Elasticsearch API

Base URL
http://api.example.com

Elasticsearch provides REST APIs that are used by the UI components and can be called directly to configure and access Elasticsearch features.

Documentation source and versions

This documentation is derived from the 8.18 branch of the elasticsearch-specification repository. It is provided under license Attribution-NonCommercial-NoDerivatives 4.0 International. This documentation contains work-in-progress information for future Elastic Stack releases.

Last update on Jun 24, 2025.

This API is provided under license Apache 2.0.


Autoscaling









Delete an autoscaling policy Generally available; Added in 7.11.0

DELETE /_autoscaling/policy/{name}

NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.

External documentation

Path parameters

  • name string Required

    the name of the autoscaling policy

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

DELETE /_autoscaling/policy/{name}
curl \
 --request DELETE 'http://api.example.com/_autoscaling/policy/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
This may be a response to either `DELETE /_autoscaling/policy/my_autoscaling_policy` or `DELETE /_autoscaling/policy/*`.
{
  "acknowledged": true
}

Get the autoscaling capacity Generally available; Added in 7.11.0

GET /_autoscaling/capacity

NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.

This API gets the current autoscaling capacity based on the configured autoscaling policy. It will return information to size the cluster appropriately to the current workload.

The required_capacity is calculated as the maximum of the required_capacity result of all individual deciders that are enabled for the policy.

The operator should verify that the current_nodes match the operator’s knowledge of the cluster to avoid making autoscaling decisions based on stale or incomplete information.

The response contains decider-specific information you can use to diagnose how and why autoscaling determined a certain capacity was required. This information is provided for diagnosis only. Do not use this information to make autoscaling decisions.

External documentation

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • policies object Required
      Hide policies attribute Show policies attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • required_capacity object Required
          Hide required_capacity attributes Show required_capacity attributes object
          • node object Required
            Hide node attributes Show node attributes object
            • storage number Required
            • memory number Required
          • total object Required
            Hide total attributes Show total attributes object
            • storage number Required
            • memory number Required
        • current_capacity object Required
          Hide current_capacity attributes Show current_capacity attributes object
          • node object Required
            Hide node attributes Show node attributes object
            • storage number Required
            • memory number Required
          • total object Required
            Hide total attributes Show total attributes object
            • storage number Required
            • memory number Required
        • current_nodes array[object] Required
          Hide current_nodes attribute Show current_nodes attribute object
          • name string Required
        • deciders object Required
          Hide deciders attribute Show deciders attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • required_capacity object Required
              Hide required_capacity attributes Show required_capacity attributes object
              • node object Required
              • total object Required
            • reason_summary string
            • reason_details object
GET /_autoscaling/capacity
curl \
 --request GET 'http://api.example.com/_autoscaling/capacity' \
 --header "Authorization: $API_KEY"
Response examples (200)
This may be a response to `GET /_autoscaling/capacity`.
{
  policies: {}
}

Get behavioral analytics collections Technical preview; Added in 8.8.0

GET /_application/analytics/{name}

Path parameters

  • name array[string] Required

    A list of analytics collections to limit the returned information

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attribute Show * attribute object
      • event_data_stream object Required
        Hide event_data_stream attribute Show event_data_stream attribute object
        • name string Required
GET /_application/analytics/{name}
curl \
 --request GET 'http://api.example.com/_application/analytics/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _application/analytics/my*`
{
  "my_analytics_collection": {
      "event_data_stream": {
          "name": "behavioral_analytics-events-my_analytics_collection"
      }
  },
  "my_analytics_collection2": {
      "event_data_stream": {
          "name": "behavioral_analytics-events-my_analytics_collection2"
      }
  }
}

Create a behavioral analytics collection Technical preview; Added in 8.8.0

PUT /_application/analytics/{name}

Path parameters

  • name string Required

    The name of the analytics collection to be created or updated.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

    • name string Required
PUT /_application/analytics/{name}
curl \
 --request PUT 'http://api.example.com/_application/analytics/{name}' \
 --header "Authorization: $API_KEY"

Delete a behavioral analytics collection Technical preview; Added in 8.8.0

DELETE /_application/analytics/{name}

The associated data stream is also deleted.

Path parameters

  • name string Required

    The name of the analytics collection to be deleted

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

DELETE /_application/analytics/{name}
curl \
 --request DELETE 'http://api.example.com/_application/analytics/{name}' \
 --header "Authorization: $API_KEY"

Get behavioral analytics collections Technical preview; Added in 8.8.0

GET /_application/analytics

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attribute Show * attribute object
      • event_data_stream object Required
        Hide event_data_stream attribute Show event_data_stream attribute object
        • name string Required
GET /_application/analytics
curl \
 --request GET 'http://api.example.com/_application/analytics' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _application/analytics/my*`
{
  "my_analytics_collection": {
      "event_data_stream": {
          "name": "behavioral_analytics-events-my_analytics_collection"
      }
  },
  "my_analytics_collection2": {
      "event_data_stream": {
          "name": "behavioral_analytics-events-my_analytics_collection2"
      }
  }
}

Create a behavioral analytics collection event Technical preview

POST /_application/analytics/{collection_name}/event/{event_type} External documentation

Path parameters

  • collection_name string Required

    The name of the behavioral analytics collection.

  • event_type string Required

    The analytics event type.

    Values are page_view, search, or search_click.

Query parameters

  • debug boolean

    Whether the response type has to include more details

application/json

Body Required

object object

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • accepted boolean Required
    • event object
POST /_application/analytics/{collection_name}/event/{event_type}
curl \
 --request POST 'http://api.example.com/_application/analytics/{collection_name}/event/{event_type}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"session\": {\n    \"id\": \"1797ca95-91c9-4e2e-b1bd-9c38e6f386a9\"\n  },\n  \"user\": {\n    \"id\": \"5f26f01a-bbee-4202-9298-81261067abbd\"\n  },\n  \"search\":{\n    \"query\": \"search term\",\n    \"results\": {\n      \"items\": [\n        {\n          \"document\": {\n            \"id\": \"123\",\n            \"index\": \"products\"\n          }\n        }\n      ],\n      \"total_results\": 10\n    },\n    \"sort\": {\n      \"name\": \"relevance\"\n    },\n    \"search_application\": \"website\"\n  },\n  \"document\":{\n    \"id\": \"123\",\n    \"index\": \"products\"\n  }\n}"'
Request example
Run `POST _application/analytics/my_analytics_collection/event/search_click` to send a `search_click` event to an analytics collection called `my_analytics_collection`.
{
  "session": {
    "id": "1797ca95-91c9-4e2e-b1bd-9c38e6f386a9"
  },
  "user": {
    "id": "5f26f01a-bbee-4202-9298-81261067abbd"
  },
  "search":{
    "query": "search term",
    "results": {
      "items": [
        {
          "document": {
            "id": "123",
            "index": "products"
          }
        }
      ],
      "total_results": 10
    },
    "sort": {
      "name": "relevance"
    },
    "search_application": "website"
  },
  "document":{
    "id": "123",
    "index": "products"
  }
}

Compact and aligned text (CAT)

The compact and aligned text (CAT) APIs aim are intended only for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, it's recommend to use a corresponding JSON API. All the cat commands accept a query string parameter help to see all the headers and info they provide, and the /_cat command alone lists all the available commands.





Get aliases Generally available

GET /_cat/aliases/{name}

Get the cluster's index aliases, including filter and routing information. This API does not return data stream aliases.

IMPORTANT: CAT APIs are only intended for human consumption using the command line or the Kibana console. They are not intended for use by applications. For application consumption, use the aliases API.

Required authorization

  • Index privileges: view_index_metadata

Path parameters

  • name string | array[string] Required

    A comma-separated list of aliases to retrieve. Supports wildcards (*). To retrieve all aliases, omit this parameter or use * or _all.

Query parameters

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • expand_wildcards string | array[string]

    The type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. It supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • local boolean

    If true, the request computes the list of selected nodes from the local cluster state. If false the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node.

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. To indicated that the request should never timeout, you can set it to -1.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • alias string

      alias name

    • index string
    • filter string

      filter

    • routing.index string

      index routing

    • is_write_index string

      write index

GET /_cat/aliases/{name}
curl \
 --request GET 'http://api.example.com/_cat/aliases/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _cat/aliases?format=json&v=true`. This response shows that `alias2` has configured a filter and `alias3` and `alias4` have routing configurations.
[
  {
    "alias": "alias1",
    "index": "test1",
    "filter": "-",
    "routing.index": "-",
    "routing.search": "-",
    "is_write_index": "true"
  },
  {
    "alias": "alias1",
    "index": "test1",
    "filter": "*",
    "routing.index": "-",
    "routing.search": "-",
    "is_write_index": "true"
  },
  {
    "alias": "alias3",
    "index": "test1",
    "filter": "-",
    "routing.index": "1",
    "routing.search": "1",
    "is_write_index": "true"
  },
  {
    "alias": "alias4",
    "index": "test1",
    "filter": "-",
    "routing.index": "2",
    "routing.search": "1,2",
    "is_write_index": "true"
  }
]




Get shard allocation information Generally available

GET /_cat/allocation/{node_id}

Get a snapshot of the number of shards allocated to each data node and their disk space.

IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications.

Required authorization

  • Cluster privileges: monitor

Path parameters

  • node_id string | array[string] Required

    A comma-separated list of node identifiers or names used to limit the returned information.

Query parameters

  • bytes string

    The unit used to display byte values.

    Values are b, kb, mb, gb, tb, or pb.

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • local boolean

    If true, the request computes the list of selected nodes from the local cluster state. If false the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node.

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

GET /_cat/allocation/{node_id}
curl \
 --request GET 'http://api.example.com/_cat/allocation/{node_id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_cat/allocation?v=true&format=json`. It shows a single shard is allocated to the one node available.
[
  {
    "shards": "1",
    "shards.undesired": "0",
    "write_load.forecast": "0.0",
    "disk.indices.forecast": "260b",
    "disk.indices": "260b",
    "disk.used": "47.3gb",
    "disk.avail": "43.4gb",
    "disk.total": "100.7gb",
    "disk.percent": "46",
    "host": "127.0.0.1",
    "ip": "127.0.0.1",
    "node": "CSUXak2",
    "node.role": "himrst"
  }
]

Get component templates Generally available; Added in 5.1.0

GET /_cat/component_templates

Get information about component templates in a cluster. Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.

IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get component template API.

Required authorization

  • Cluster privileges: monitor

Query parameters

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • local boolean

    If true, the request computes the list of selected nodes from the local cluster state. If false the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node.

  • master_timeout string

    The period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • name string Required
    • version string | null Required

    • alias_count string Required
    • mapping_count string Required
    • settings_count string Required
    • metadata_count string Required
    • included_in string Required
GET /_cat/component_templates
curl \
 --request GET 'http://api.example.com/_cat/component_templates' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _cat/component_templates/my-template-*?v=true&s=name&format=json`.
[
  {
    "name": "my-template-1",
    "version": "null",
    "alias_count": "0",
    "mapping_count": "0",
    "settings_count": "1",
    "metadata_count": "0",
    "included_in": "[my-index-template]"
  },
    {
    "name": "my-template-2",
    "version": null,
    "alias_count": "0",
    "mapping_count": "3",
    "settings_count": "0",
    "metadata_count": "0",
    "included_in": "[my-index-template]"
  }
]

Get component templates Generally available; Added in 5.1.0

GET /_cat/component_templates/{name}

Get information about component templates in a cluster. Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.

IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get component template API.

Required authorization

  • Cluster privileges: monitor

Path parameters

  • name string Required

    The name of the component template. It accepts wildcard expressions. If it is omitted, all component templates are returned.

Query parameters

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • local boolean

    If true, the request computes the list of selected nodes from the local cluster state. If false the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node.

  • master_timeout string

    The period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • name string Required
    • version string | null Required

    • alias_count string Required
    • mapping_count string Required
    • settings_count string Required
    • metadata_count string Required
    • included_in string Required
GET /_cat/component_templates/{name}
curl \
 --request GET 'http://api.example.com/_cat/component_templates/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _cat/component_templates/my-template-*?v=true&s=name&format=json`.
[
  {
    "name": "my-template-1",
    "version": "null",
    "alias_count": "0",
    "mapping_count": "0",
    "settings_count": "1",
    "metadata_count": "0",
    "included_in": "[my-index-template]"
  },
    {
    "name": "my-template-2",
    "version": null,
    "alias_count": "0",
    "mapping_count": "3",
    "settings_count": "0",
    "metadata_count": "0",
    "included_in": "[my-index-template]"
  }
]
















Get the cluster health status Generally available

GET /_cat/health

IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the cluster health API. This API is often used to check malfunctioning clusters. To help you track cluster health alongside log files and alerting systems, the API returns timestamps in two formats: HH:MM:SS, which is human-readable but includes no date information; Unix epoch time, which is machine-sortable and includes date information. The latter format is useful for cluster recoveries that take multiple days. You can use the cat health API to verify cluster health across multiple nodes. You also can use the API to track the recovery of a large cluster over a longer period of time.

Required authorization

  • Cluster privileges: monitor

Query parameters

  • time string

    The unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

  • ts boolean

    If true, returns HH:MM:SS and Unix epoch timestamps.

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • epoch number | string

      Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

      Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

    • timestamp string

      Time of day, expressed as HH:MM:SS

    • cluster string

      cluster name

    • status string

      health status

    • node.total string

      total number of nodes

    • node.data string

      number of nodes that can store data

    • shards string

      total number of shards

    • pri string

      number of primary shards

    • relo string

      number of relocating nodes

    • init string

      number of initializing nodes

    • unassign.pri string

      number of unassigned primary shards

    • unassign string

      number of unassigned shards

    • pending_tasks string

      number of pending tasks

    • max_task_wait_time string

      wait time of longest task pending

    • active_shards_percent string

      active number of shards in percent

GET /_cat/health
curl \
 --request GET 'http://api.example.com/_cat/health' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_cat/health?v=true&format=json`. By default, it returns `HH:MM:SS` and Unix epoch timestamps.
[
  {
    "epoch": "1475871424",
    "timestamp": "16:17:04",
    "cluster": "elasticsearch",
    "status": "green",
    "node.total": "1",
    "node.data": "1",
    "shards": "1",
    "pri": "1",
    "relo": "0",
    "init": "0",
    "unassign": "0",
    "unassign.pri": "0",
    "pending_tasks": "0",
    "max_task_wait_time": "-",
    "active_shards_percent": "100.0%"
  }
]

Get CAT help Generally available

GET /_cat

Get help for the CAT APIs.

Responses

  • 200 application/json
GET /_cat
curl \
 --request GET 'http://api.example.com/_cat' \
 --header "Authorization: $API_KEY"








Get master node information Generally available

GET /_cat/master

Get information about the master node, including the ID, bound IP address, and name.

IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes info API.

Required authorization

  • Cluster privileges: monitor

Query parameters

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • local boolean

    If true, the request computes the list of selected nodes from the local cluster state. If false the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node.

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string

      node id

    • host string

      host name

    • ip string

      ip address

    • node string

      node name

GET /_cat/master
curl \
 --request GET 'http://api.example.com/_cat/master' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_cat/master?v=true&format=json`.
[
  {
    "id": "YzWoH_2BT-6UjVGDyPdqYg",
    "host": "127.0.0.1",
    "ip": "127.0.0.1",
    "node": "YzWoH_2"
  }
]

Get data frame analytics jobs Generally available; Added in 7.7.0

GET /_cat/ml/data_frame/analytics

Get configuration and usage information about data frame analytics jobs.

IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get data frame analytics jobs statistics API.

Required authorization

  • Cluster privileges: monitor_ml

Query parameters

  • allow_no_match boolean

    Whether to ignore if a wildcard expression matches no configs. (This includes _all string or when no configs have been specified)

  • bytes string

    The unit in which to display byte values

    Values are b, kb, mb, gb, tb, or pb.

  • h string | array[string]

    Comma-separated list of column names to display.

  • s string | array[string]

    Comma-separated list of column names or column aliases used to sort the response.

  • time string

    Unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string
    • type string

      The type of analysis that the job performs.

    • create_time string

      The time when the job was created.

    • version string
    • source_index string
    • dest_index string
    • description string

      A description of the job.

    • model_memory_limit string

      The approximate maximum amount of memory resources that are permitted for the job.

    • state string

      The current status of the job.

    • failure_reason string

      Messages about the reason why the job failed.

    • progress string

      The progress report for the job by phase.

    • assignment_explanation string

      Messages related to the selection of a node.

    • node.id string
    • node.name string
    • node.ephemeral_id string
    • node.address string

      The network address of the assigned node.

GET /_cat/ml/data_frame/analytics
curl \
 --request GET 'http://api.example.com/_cat/ml/data_frame/analytics' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _cat/ml/data_frame/analytics?v=true&format=json`.
[
  {
    "id": "classifier_job_1",
    "type": "classification",
    "create_time": "2020-02-12T11:49:09.594Z",
    "state": "stopped"
  },
    {
    "id": "classifier_job_2",
    "type": "classification",
    "create_time": "2020-02-12T11:49:14.479Z",
    "state": "stopped"
  },
  {
    "id": "classifier_job_3",
    "type": "classification",
    "create_time": "2020-02-12T11:49:16.928Z",
    "state": "stopped"
  },
  {
    "id": "classifier_job_4",
    "type": "classification",
    "create_time": "2020-02-12T11:49:19.127Z",
    "state": "stopped"
  },
  {
    "id": "classifier_job_5",
    "type": "classification",
    "create_time": "2020-02-12T11:49:21.349Z",
    "state": "stopped"
  }
]




Get datafeeds Generally available; Added in 7.7.0

GET /_cat/ml/datafeeds

Get configuration and usage information about datafeeds. This API returns a maximum of 10,000 datafeeds. If the Elasticsearch security features are enabled, you must have monitor_ml, monitor, manage_ml, or manage cluster privileges to use this API.

IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get datafeed statistics API.

Required authorization

  • Cluster privileges: monitor_ml

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    • Contains wildcard expressions and there are no datafeeds that match.
    • Contains the _all string or no identifiers and there are no matches.
    • Contains wildcard expressions and there are only partial matches.

    If true, the API returns an empty datafeeds array when there are no matches and the subset of results when there are partial matches. If false, the API returns a 404 status code when there are no matches or only partial matches.

  • h string | array[string]

    Comma-separated list of column names to display.

  • s string | array[string]

    Comma-separated list of column names or column aliases used to sort the response.

  • time string

    The unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string

      The datafeed identifier.

    • state string

      Values are started, stopped, starting, or stopping.

    • assignment_explanation string

      For started datafeeds only, contains messages relating to the selection of a node.

    • buckets.count string

      The number of buckets processed.

    • search.count string

      The number of searches run by the datafeed.

    • search.time string

      The total time the datafeed spent searching, in milliseconds.

    • search.bucket_avg string

      The average search time per bucket, in milliseconds.

    • search.exp_avg_hour string

      The exponential average search time per hour, in milliseconds.

    • node.id string

      The unique identifier of the assigned node. For started datafeeds only, this information pertains to the node upon which the datafeed is started.

    • node.name string

      The name of the assigned node. For started datafeeds only, this information pertains to the node upon which the datafeed is started.

    • node.ephemeral_id string

      The ephemeral identifier of the assigned node. For started datafeeds only, this information pertains to the node upon which the datafeed is started.

    • node.address string

      The network address of the assigned node. For started datafeeds only, this information pertains to the node upon which the datafeed is started.

GET /_cat/ml/datafeeds
curl \
 --request GET 'http://api.example.com/_cat/ml/datafeeds' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _cat/ml/datafeeds?v=true&format=json`.
[
  {
    "id": "datafeed-high_sum_total_sales",
    "state": "stopped",
    "buckets.count": "743",
    "search.count": "7"
  },
  {
    "id": "datafeed-low_request_rate",
    "state": "stopped",
    "buckets.count": "1457",
    "search.count": "3"
  },
  {
    "id": "datafeed-response_code_rates",
    "state": "stopped",
    "buckets.count": "1460",
    "search.count": "18"
  },
  {
    "id": "datafeed-url_scanning",
    "state": "stopped",
    "buckets.count": "1460",
    "search.count": "18"
  }
]




Get anomaly detection jobs Generally available; Added in 7.7.0

GET /_cat/ml/anomaly_detectors

Get configuration and usage information for anomaly detection jobs. This API returns a maximum of 10,000 jobs. If the Elasticsearch security features are enabled, you must have monitor_ml, monitor, manage_ml, or manage cluster privileges to use this API.

IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get anomaly detection job statistics API.

Required authorization

  • Cluster privileges: monitor_ml

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    • Contains wildcard expressions and there are no jobs that match.
    • Contains the _all string or no identifiers and there are no matches.
    • Contains wildcard expressions and there are only partial matches.

    If true, the API returns an empty jobs array when there are no matches and the subset of results when there are partial matches. If false, the API returns a 404 status code when there are no matches or only partial matches.

  • bytes string

    The unit used to display byte values.

    Values are b, kb, mb, gb, tb, or pb.

  • h string | array[string]

    Comma-separated list of column names to display.

  • s string | array[string]

    Comma-separated list of column names or column aliases used to sort the response.

  • time string

    The unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string
    • state string

      Values are closing, closed, opened, failed, or opening.

    • opened_time string

      For open jobs only, the amount of time the job has been opened.

    • assignment_explanation string

      For open anomaly detection jobs only, contains messages relating to the selection of a node to run the job.

    • data.processed_records string

      The number of input documents that have been processed by the anomaly detection job. This value includes documents with missing fields, since they are nonetheless analyzed. If you use datafeeds and have aggregations in your search query, the processed_record_count is the number of aggregation results processed, not the number of Elasticsearch documents.

    • data.processed_fields string

      The total number of fields in all the documents that have been processed by the anomaly detection job. Only fields that are specified in the detector configuration object contribute to this count. The timestamp is not included in this count.

    • data.input_bytes number | string

    • data.input_records string

      The number of input documents posted to the anomaly detection job.

    • data.input_fields string

      The total number of fields in input documents posted to the anomaly detection job. This count includes fields that are not used in the analysis. However, be aware that if you are using a datafeed, it extracts only the required fields from the documents it retrieves before posting them to the job.

    • data.invalid_dates string

      The number of input documents with either a missing date field or a date that could not be parsed.

    • data.missing_fields string

      The number of input documents that are missing a field that the anomaly detection job is configured to analyze. Input documents with missing fields are still processed because it is possible that not all fields are missing. If you are using datafeeds or posting data to the job in JSON format, a high missing_field_count is often not an indication of data issues. It is not necessarily a cause for concern.

    • data.out_of_order_timestamps string

      The number of input documents that have a timestamp chronologically preceding the start of the current anomaly detection bucket offset by the latency window. This information is applicable only when you provide data to the anomaly detection job by using the post data API. These out of order documents are discarded, since jobs require time series data to be in ascending chronological order.

    • data.empty_buckets string

      The number of buckets which did not contain any data. If your data contains many empty buckets, consider increasing your bucket_span or using functions that are tolerant to gaps in data such as mean, non_null_sum or non_zero_count.

    • data.sparse_buckets string

      The number of buckets that contained few data points compared to the expected number of data points. If your data contains many sparse buckets, consider using a longer bucket_span.

    • data.buckets string

      The total number of buckets processed.

    • data.earliest_record string

      The timestamp of the earliest chronologically input document.

    • data.latest_record string

      The timestamp of the latest chronologically input document.

    • data.last string

      The timestamp at which data was last analyzed, according to server time.

    • data.last_empty_bucket string

      The timestamp of the last bucket that did not contain any data.

    • data.last_sparse_bucket string

      The timestamp of the last bucket that was considered sparse.

    • model.bytes number | string

    • model.memory_status string

      Values are ok, soft_limit, or hard_limit.

    • model.bytes_exceeded number | string

    • model.memory_limit string

      The upper limit for model memory usage, checked on increasing values.

    • model.by_fields string

      The number of by field values that were analyzed by the models. This value is cumulative for all detectors in the job.

    • model.over_fields string

      The number of over field values that were analyzed by the models. This value is cumulative for all detectors in the job.

    • model.partition_fields string

      The number of partition field values that were analyzed by the models. This value is cumulative for all detectors in the job.

    • model.bucket_allocation_failures string

      The number of buckets for which new entities in incoming data were not processed due to insufficient model memory. This situation is also signified by a hard_limit: memory_status property value.

    • model.categorization_status string

      Values are ok or warn.

    • model.categorized_doc_count string

      The number of documents that have had a field categorized.

    • model.total_category_count string

      The number of categories created by categorization.

    • model.frequent_category_count string

      The number of categories that match more than 1% of categorized documents.

    • model.rare_category_count string

      The number of categories that match just one categorized document.

    • model.dead_category_count string

      The number of categories created by categorization that will never be assigned again because another category’s definition makes it a superset of the dead category. Dead categories are a side effect of the way categorization has no prior training.

    • model.failed_category_count string

      The number of times that categorization wanted to create a new category but couldn’t because the job had hit its model_memory_limit. This count does not track which specific categories failed to be created. Therefore you cannot use this value to determine the number of unique categories that were missed.

    • model.log_time string

      The timestamp when the model stats were gathered, according to server time.

    • model.timestamp string

      The timestamp of the last record when the model stats were gathered.

    • forecasts.total string

      The number of individual forecasts currently available for the job. A value of one or more indicates that forecasts exist.

    • forecasts.memory.min string

      The minimum memory usage in bytes for forecasts related to the anomaly detection job.

    • forecasts.memory.max string

      The maximum memory usage in bytes for forecasts related to the anomaly detection job.

    • forecasts.memory.avg string

      The average memory usage in bytes for forecasts related to the anomaly detection job.

    • forecasts.memory.total string

      The total memory usage in bytes for forecasts related to the anomaly detection job.

    • forecasts.records.min string

      The minimum number of model_forecast documents written for forecasts related to the anomaly detection job.

    • forecasts.records.max string

      The maximum number of model_forecast documents written for forecasts related to the anomaly detection job.

    • forecasts.records.avg string

      The average number of model_forecast documents written for forecasts related to the anomaly detection job.

    • forecasts.records.total string

      The total number of model_forecast documents written for forecasts related to the anomaly detection job.

    • forecasts.time.min string

      The minimum runtime in milliseconds for forecasts related to the anomaly detection job.

    • forecasts.time.max string

      The maximum runtime in milliseconds for forecasts related to the anomaly detection job.

    • forecasts.time.avg string

      The average runtime in milliseconds for forecasts related to the anomaly detection job.

    • forecasts.time.total string

      The total runtime in milliseconds for forecasts related to the anomaly detection job.

    • node.id string
    • node.name string

      The name of the assigned node.

    • node.ephemeral_id string
    • node.address string

      The network address of the assigned node.

    • buckets.count string

      The number of bucket results produced by the job.

    • buckets.time.total string

      The sum of all bucket processing times, in milliseconds.

    • buckets.time.min string

      The minimum of all bucket processing times, in milliseconds.

    • buckets.time.max string

      The maximum of all bucket processing times, in milliseconds.

    • buckets.time.exp_avg string

      The exponential moving average of all bucket processing times, in milliseconds.

    • buckets.time.exp_avg_hour string

      The exponential moving average of bucket processing times calculated in a one hour time window, in milliseconds.

GET /_cat/ml/anomaly_detectors
curl \
 --request GET 'http://api.example.com/_cat/ml/anomaly_detectors' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _cat/ml/anomaly_detectors?h=id,s,dpr,mb&v=true&format=json`.
[
  {
    "id": "high_sum_total_sales",
    "s": "closed",
    "dpr": "14022",
    "mb": "1.5mb"
  },
  {
    "id": "low_request_rate",
    "s": "closed",
    "dpr": "1216",
    "mb": "40.5kb"
  },
  {
    "id": "response_code_rates",
    "s": "closed",
    "dpr": "28146",
    "mb": "132.7kb"
  },
  {
    "id": "url_scanning",
    "s": "closed",
    "dpr": "28146",
    "mb": "501.6kb"
  }
]












Get node attribute information Generally available

GET /_cat/nodeattrs

Get information about custom node attributes. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes info API.

Required authorization

  • Cluster privileges: monitor

Query parameters

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • local boolean

    If true, the request computes the list of selected nodes from the local cluster state. If false the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node.

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • node string

      The node name.

    • id string

      The unique node identifier.

    • pid string

      The process identifier.

    • host string

      The host name.

    • ip string

      The IP address.

    • port string

      The bound transport port.

    • attr string

      The attribute name.

    • value string

      The attribute value.

GET /_cat/nodeattrs
curl \
 --request GET 'http://api.example.com/_cat/nodeattrs' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_cat/nodeattrs?v=true&format=json`. The `node`, `host`, and `ip` columns provide basic information about each node. The `attr` and `value` columns return custom node attributes, one per line.
[
  {
    "node": "node-0",
    "host": "127.0.0.1",
    "ip": "127.0.0.1",
    "attr": "testattr",
    "value": "test"
  }
]
A successful response from `GET /_cat/nodeattrs?v=true&h=name,pid,attr,value`. It returns the `name`, `pid`, `attr`, and `value` columns.
[
  {
    "name": "node-0",
    "pid": "19566",
    "attr": "testattr",
    "value": "test"
  }
]












Get shard recovery information Generally available

GET /_cat/recovery

Get information about ongoing and completed shard recoveries. Shard recovery is the process of initializing a shard copy, such as restoring a primary shard from a snapshot or syncing a replica shard from a primary shard. When a shard recovery completes, the recovered shard is available for search and indexing. For data streams, the API returns information about the stream’s backing indices. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the index recovery API.

Required authorization

  • Index privileges: monitor
  • Cluster privileges: monitor

Query parameters

  • active_only boolean

    If true, the response only includes ongoing shard recoveries.

  • bytes string

    The unit used to display byte values.

    Values are b, kb, mb, gb, tb, or pb.

  • detailed boolean

    If true, the response includes detailed information about shard recoveries.

  • index string | array[string]

    Comma-separated list or wildcard expression of index names to limit the returned information

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • time string

    Unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • index string
    • shard string

      The shard name.

    • start_time string | number

      A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

      One of:
    • start_time_millis number

      Time unit for milliseconds

    • stop_time string | number

      A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

      One of:
    • stop_time_millis number

      Time unit for milliseconds

    • time string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • type string

      The recovery type.

    • stage string

      The recovery stage.

    • source_host string

      The source host.

    • source_node string

      The source node name.

    • target_host string

      The target host.

    • target_node string

      The target node name.

    • repository string

      The repository name.

    • snapshot string

      The snapshot name.

    • files string

      The number of files to recover.

    • files_recovered string

      The files recovered.

    • files_percent string | number

    • files_total string

      The total number of files.

    • bytes string

      The number of bytes to recover.

    • bytes_recovered string

      The bytes recovered.

    • bytes_percent string | number

    • bytes_total string

      The total number of bytes.

    • translog_ops string

      The number of translog operations to recover.

    • translog_ops_recovered string

      The translog operations recovered.

    • translog_ops_percent string | number

GET /_cat/recovery
curl \
 --request GET 'http://api.example.com/_cat/recovery' \
 --header "Authorization: $API_KEY"
A successful response from `GET _cat/recovery?v=true&format=json`. In this example, the source and target nodes are the same because the recovery type is `store`, meaning they were read from local storage on node start.
[
  {
    "index": "my-index-000001 ",
    "shard": "0",
    "time": "13ms",
    "type": "store",
    "stage": "done",
    "source_host": "n/a",
    "source_node": "n/a",
    "target_host": "127.0.0.1",
    "target_node": "node-0",
    "repository": "n/a",
    "snapshot": "n/a",
    "files": "0",
    "files_recovered": "0",
    "files_percent": "100.0%",
    "files_total": "13",
    "bytes": "0b",
    "bytes_recovered": "0b",
    "bytes_percent": "100.0%",
    "bytes_total": "9928b",
    "translog_ops": "0",
    "translog_ops_recovered": "0",
    "translog_ops_percent": "100.0%"
  }
]
A successful response from `GET _cat/recovery?v=true&h=i,s,t,ty,st,shost,thost,f,fp,b,bp&format=json`. You can retrieve information about an ongoing recovery for example when you increase the replica count of an index and bring another node online to host the replicas. In this example, the recovery type is `peer`, meaning the shard recovered from another node. The `files` and `bytes` are real-time measurements.
[
  {
    "i": "my-index-000001",
    "s": "0",
    "t": "1252ms",
    "ty": "peer",
    "st": "done",
    "shost": "192.168.1.1",
    "thost": "192.168.1.1",
    "f": "0",
    "fp": "100.0%",
    "b": "0b",
    "bp": "100.0%",
  }
]
A successful response from `GET _cat/recovery?v=true&h=i,s,t,ty,st,rep,snap,f,fp,b,bp&format=json`. You can restore backups of an index using the snapshot and restore API. You can use the cat recovery API to get information about a snapshot recovery.
[
  {
    "i": "my-index-000001",
    "s": "0",
    "t": "1978ms",
    "ty": "snapshot",
    "st": "done",
    "rep": "my-repo",
    "snap": "snap-1",
    "f": "79",
    "fp": "8.0%",
    "b": "12086",
    "bp": "9.0%"
  }
]

Get shard recovery information Generally available

GET /_cat/recovery/{index}

Get information about ongoing and completed shard recoveries. Shard recovery is the process of initializing a shard copy, such as restoring a primary shard from a snapshot or syncing a replica shard from a primary shard. When a shard recovery completes, the recovered shard is available for search and indexing. For data streams, the API returns information about the stream’s backing indices. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the index recovery API.

Required authorization

  • Index privileges: monitor
  • Cluster privileges: monitor

Path parameters

  • index string | array[string] Required

    A comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • active_only boolean

    If true, the response only includes ongoing shard recoveries.

  • bytes string

    The unit used to display byte values.

    Values are b, kb, mb, gb, tb, or pb.

  • detailed boolean

    If true, the response includes detailed information about shard recoveries.

  • index string | array[string]

    Comma-separated list or wildcard expression of index names to limit the returned information

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • time string

    Unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • index string
    • shard string

      The shard name.

    • start_time string | number

      A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

      One of:
    • start_time_millis number

      Time unit for milliseconds

    • stop_time string | number

      A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

      One of:
    • stop_time_millis number

      Time unit for milliseconds

    • time string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • type string

      The recovery type.

    • stage string

      The recovery stage.

    • source_host string

      The source host.

    • source_node string

      The source node name.

    • target_host string

      The target host.

    • target_node string

      The target node name.

    • repository string

      The repository name.

    • snapshot string

      The snapshot name.

    • files string

      The number of files to recover.

    • files_recovered string

      The files recovered.

    • files_percent string | number

    • files_total string

      The total number of files.

    • bytes string

      The number of bytes to recover.

    • bytes_recovered string

      The bytes recovered.

    • bytes_percent string | number

    • bytes_total string

      The total number of bytes.

    • translog_ops string

      The number of translog operations to recover.

    • translog_ops_recovered string

      The translog operations recovered.

    • translog_ops_percent string | number

GET /_cat/recovery/{index}
curl \
 --request GET 'http://api.example.com/_cat/recovery/{index}' \
 --header "Authorization: $API_KEY"
A successful response from `GET _cat/recovery?v=true&format=json`. In this example, the source and target nodes are the same because the recovery type is `store`, meaning they were read from local storage on node start.
[
  {
    "index": "my-index-000001 ",
    "shard": "0",
    "time": "13ms",
    "type": "store",
    "stage": "done",
    "source_host": "n/a",
    "source_node": "n/a",
    "target_host": "127.0.0.1",
    "target_node": "node-0",
    "repository": "n/a",
    "snapshot": "n/a",
    "files": "0",
    "files_recovered": "0",
    "files_percent": "100.0%",
    "files_total": "13",
    "bytes": "0b",
    "bytes_recovered": "0b",
    "bytes_percent": "100.0%",
    "bytes_total": "9928b",
    "translog_ops": "0",
    "translog_ops_recovered": "0",
    "translog_ops_percent": "100.0%"
  }
]
A successful response from `GET _cat/recovery?v=true&h=i,s,t,ty,st,shost,thost,f,fp,b,bp&format=json`. You can retrieve information about an ongoing recovery for example when you increase the replica count of an index and bring another node online to host the replicas. In this example, the recovery type is `peer`, meaning the shard recovered from another node. The `files` and `bytes` are real-time measurements.
[
  {
    "i": "my-index-000001",
    "s": "0",
    "t": "1252ms",
    "ty": "peer",
    "st": "done",
    "shost": "192.168.1.1",
    "thost": "192.168.1.1",
    "f": "0",
    "fp": "100.0%",
    "b": "0b",
    "bp": "100.0%",
  }
]
A successful response from `GET _cat/recovery?v=true&h=i,s,t,ty,st,rep,snap,f,fp,b,bp&format=json`. You can restore backups of an index using the snapshot and restore API. You can use the cat recovery API to get information about a snapshot recovery.
[
  {
    "i": "my-index-000001",
    "s": "0",
    "t": "1978ms",
    "ty": "snapshot",
    "st": "done",
    "rep": "my-repo",
    "snap": "snap-1",
    "f": "79",
    "fp": "8.0%",
    "b": "12086",
    "bp": "9.0%"
  }
]








Get segment information Generally available

GET /_cat/segments/{index}

Get low-level information about the Lucene segments in index shards. For data streams, the API returns information about the backing indices. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the index segments API.

Required authorization

  • Index privileges: monitor
  • Cluster privileges: monitor

Path parameters

  • index string | array[string] Required

    A comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • bytes string

    The unit used to display byte values.

    Values are b, kb, mb, gb, tb, or pb.

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • local boolean

    If true, the request computes the list of selected nodes from the local cluster state. If false the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node.

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • index string
    • shard string

      The shard name.

    • prirep string

      The shard type: primary or replica.

    • ip string

      The IP address of the node where it lives.

    • id string
    • segment string

      The segment name, which is derived from the segment generation and used internally to create file names in the directory of the shard.

    • generation string

      The segment generation number. Elasticsearch increments this generation number for each segment written then uses this number to derive the segment name.

    • docs.count string

      The number of documents in the segment. This excludes deleted documents and counts any nested documents separately from their parents. It also excludes documents which were indexed recently and do not yet belong to a segment.

    • docs.deleted string

      The number of deleted documents in the segment, which might be higher or lower than the number of delete operations you have performed. This number excludes deletes that were performed recently and do not yet belong to a segment. Deleted documents are cleaned up by the automatic merge process if it makes sense to do so. Also, Elasticsearch creates extra deleted documents to internally track the recent history of operations on a shard.

    • size number | string

    • size.memory number | string

    • committed string

      If true, the segment is synced to disk. Segments that are synced can survive a hard reboot. If false, the data from uncommitted segments is also stored in the transaction log so that Elasticsearch is able to replay changes on the next start.

    • searchable string

      If true, the segment is searchable. If false, the segment has most likely been written to disk but needs a refresh to be searchable.

    • version string
    • compound string

      If true, the segment is stored in a compound file. This means Lucene merged all files from the segment in a single file to save file descriptors.

GET /_cat/segments/{index}
curl \
 --request GET 'http://api.example.com/_cat/segments/{index}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_cat/segments?v=true&format=json`.
[
  {
    "index": "test",
    "shard": "0",
    "prirep": "p",
    "ip": "127.0.0.1",
    "segment": "_0",
    "generation": "0",
    "docs.count": "1",
    "docs.deleted": "0",
    "size": "3kb",
    "size.memory": "0",
    "committed": "false",
    "searchable": "true",
    "version": "9.12.0",
    "compound": "true"
  },
  {
    "index": "test1",
    "shard": "0",
    "prirep": "p",
    "ip": "127.0.0.1",
    "segment": "_0",
    "generation": "0",
    "docs.count": "1",
    "docs.deleted": "0",
    "size": "3kb",
    "size.memory": "0",
    "committed": "false",
    "searchable": "true",
    "version": "9.12.0",
    "compound": "true"
  }
]








Get snapshot information Generally available; Added in 2.1.0

GET /_cat/snapshots

Get information about the snapshots stored in one or more repositories. A snapshot is a backup of an index or running Elasticsearch cluster. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get snapshot API.

Required authorization

  • Cluster privileges: monitor_snapshot

Query parameters

  • ignore_unavailable boolean

    If true, the response does not include information from unavailable snapshots.

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

  • time string

    Unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string

      The unique identifier for the snapshot.

    • repository string

      The repository name.

    • status string

      The state of the snapshot process. Returned values include: FAILED: The snapshot process failed. INCOMPATIBLE: The snapshot process is incompatible with the current cluster version. IN_PROGRESS: The snapshot process started but has not completed. PARTIAL: The snapshot process completed with a partial success. SUCCESS: The snapshot process completed with a full success.

    • start_epoch number | string

      Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

      Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

    • start_time string | object

      A time of day, expressed either as hh:mm, noon, midnight, or an hour/minutes structure.

      One of:
    • end_epoch number | string

      Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

      Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

    • end_time string

      Time of day, expressed as HH:MM:SS

    • duration string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • indices string

      The number of indices in the snapshot.

    • successful_shards string

      The number of successful shards in the snapshot.

    • failed_shards string

      The number of failed shards in the snapshot.

    • total_shards string

      The total number of shards in the snapshot.

    • reason string

      The reason for any snapshot failures.

GET /_cat/snapshots
curl \
 --request GET 'http://api.example.com/_cat/snapshots' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_cat/snapshots/repo1?v=true&s=id&format=json`.
[
  {
    "id": "snap1",
    "repository": "repo1",
    "status": "FAILED",
    "start_epoch": "1445616705",
    "start_time": "18:11:45",
    "end_epoch": "1445616978",
    "end_time": "18:16:18",
    "duration": "4.6m",
    "indices": "1",
    "successful_shards": "4",
    "failed_shards": "1",
    "total_shards": "5"
  },
  {
    "id": "snap2",
    "repository": "repo1",
    "status": "SUCCESS",
    "start_epoch": "1445634298",
    "start_time": "23:04:58",
    "end_epoch": "1445634672",
    "end_time": "23:11:12",
    "duration": "6.2m",
    "indices": "2",
    "successful_shards": "10",
    "failed_shards": "0",
    "total_shards": "10"
  }
]




Get task information Technical preview; Added in 5.0.0

GET /_cat/tasks

Get information about tasks currently running in the cluster. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the task management API.

Required authorization

  • Cluster privileges: monitor

Query parameters

  • actions array[string]

    The task action names, which are used to limit the response.

  • detailed boolean

    If true, the response includes detailed information about shard recoveries.

  • nodes array[string]

    Unique node identifiers, which are used to limit the response.

  • parent_task_id string

    The parent task identifier, which is used to limit the response.

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • time string

    Unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • wait_for_completion boolean

    If true, the request blocks until the task has completed.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string
    • action string

      The task action.

    • task_id string
    • parent_task_id string

      The parent task identifier.

    • type string

      The task type.

    • start_time string

      The start time in milliseconds.

    • timestamp string

      The start time in HH:MM:SS format.

    • running_time_ns string

      The running time in nanoseconds.

    • running_time string

      The running time.

    • node_id string
    • ip string

      The IP address for the node.

    • port string

      The bound transport port for the node.

    • node string

      The node name.

    • version string
    • x_opaque_id string

      The X-Opaque-ID header.

    • description string

      The task action description.

GET /_cat/tasks
curl \
 --request GET 'http://api.example.com/_cat/tasks' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _cat/tasks?v=true&format=json`.
[
  {
    "action": "cluster:monitor/tasks/lists[n]",
    "task_id": "oTUltX4IQMOUUVeiohTt8A:124",
    "parent_task_id": "oTUltX4IQMOUUVeiohTt8A:123",
    "type": "direct",
    "start_time": "1458585884904",
    "timestamp": "01:48:24",
    "running_time": "44.1micros",
    "ip": "127.0.0.1:9300",
    "node": "oTUltX4IQMOUUVeiohTt8A"
  },
  {
    "action": "cluster:monitor/tasks/lists",
    "task_id": "oTUltX4IQMOUUVeiohTt8A:123",
    "parent_task_id": "-",
    "type": "transport",
    "start_time": "1458585884904",
    "timestamp": "01:48:24",
    "running_time": "186.2micros",
    "ip": "127.0.0.1:9300",
    "node": "oTUltX4IQMOUUVeiohTt8A"
  }
]








Get thread pool statistics Generally available

GET /_cat/thread_pool

Get thread pool statistics for each node in a cluster. Returned information includes all built-in thread pools and custom thread pools. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes info API.

Required authorization

  • Cluster privileges: monitor

Query parameters

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

  • time string

    The unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

  • local boolean

    If true, the request computes the list of selected nodes from the local cluster state. If false the list of selected nodes are computed from the cluster state of the master node. In both cases the coordinating node will send requests for further information to each selected node.

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • node_name string

      The node name.

    • node_id string
    • ephemeral_node_id string

      The ephemeral node identifier.

    • pid string

      The process identifier.

    • host string

      The host name for the current node.

    • ip string

      The IP address for the current node.

    • port string

      The bound transport port for the current node.

    • name string

      The thread pool name.

    • type string

      The thread pool type. Returned values include fixed, fixed_auto_queue_size, direct, and scaling.

    • active string

      The number of active threads in the current thread pool.

    • pool_size string

      The number of threads in the current thread pool.

    • queue string

      The number of tasks currently in queue.

    • queue_size string

      The maximum number of tasks permitted in the queue.

    • rejected string

      The number of rejected tasks.

    • largest string

      The highest number of active threads in the current thread pool.

    • completed string

      The number of completed tasks.

    • core string | null

      The core number of active threads allowed in a scaling thread pool.

    • max string | null

      The maximum number of active threads allowed in a scaling thread pool.

    • size string | null

      The number of active threads allowed in a fixed thread pool.

    • keep_alive string | null

      The thread keep alive time.

GET /_cat/thread_pool
curl \
 --request GET 'http://api.example.com/_cat/thread_pool' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_cat/thread_pool?format=json`.
[
  {
    "node_name": "node-0",
    "name": "analyze",
    "active": "0",
    "queue": "0",
    "rejected": "0"
  },
  {
    "node_name": "node-0",
    "name": "fetch_shard_started",
    "active": "0",
    "queue": "0",
    "rejected": "0"
  },
  {
    "node_name": "node-0",
    "name": "fetch_shard_store",
    "active": "0",
    "queue": "0",
    "rejected": "0"
  },
  {
    "node_name": "node-0",
    "name": "flush",
    "active": "0",
    "queue": "0",
    "rejected": "0"
  },
  {
    "node_name": "node-0",
    "name": "write",
    "active": "0",
    "queue": "0",
    "rejected": "0"
  }
]
A successful response from `GET /_cat/thread_pool/generic?v=true&h=id,name,active,rejected,completed&format=json`. It returns the `id`, `name`, `active`, `rejected`, and `completed` columns. It also limits returned information to the generic thread pool.
[
  {
    "id": "0EWUhXeBQtaVGlexUeVwMg",
    "name": "generic",
    "active": "0",
    "rejected": "0",
    "completed": "70"
  }
]












Cluster

Explain the shard allocations Generally available; Added in 5.0.0

GET /_cluster/allocation/explain

Get explanations for shard allocations in the cluster. For unassigned shards, it provides an explanation for why the shard is unassigned. For assigned shards, it provides an explanation for why the shard is remaining on its current node and has not moved or rebalanced to another node. This API can be very useful when attempting to diagnose why a shard is unassigned or why a shard continues to remain on its current node when you might expect otherwise.

Query parameters

  • include_disk_info boolean

    If true, returns information about disk usage and shard sizes.

  • include_yes_decisions boolean

    If true, returns YES decisions in explanation.

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

application/json

Body

  • current_node string

    Specifies the node ID or the name of the node to only explain a shard that is currently located on the specified node.

  • index string
  • primary boolean

    If true, returns explanation for the primary shard for the given shard ID.

  • shard number

    Specifies the ID of the shard that you would like an explanation for.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • allocate_explanation string
    • allocation_delay string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • allocation_delay_in_millis number

      Time unit for milliseconds

    • can_allocate string

      Values are yes, no, worse_balance, throttled, awaiting_info, allocation_delayed, no_valid_shard_copy, or no_attempt.

    • can_move_to_other_node string

      Values are yes, no, worse_balance, throttled, awaiting_info, allocation_delayed, no_valid_shard_copy, or no_attempt.

    • can_rebalance_cluster string

      Values are yes, no, worse_balance, throttled, awaiting_info, allocation_delayed, no_valid_shard_copy, or no_attempt.

    • can_rebalance_cluster_decisions array[object]
      Hide can_rebalance_cluster_decisions attributes Show can_rebalance_cluster_decisions attributes object
      • decider string Required
      • decision string Required

        Values are NO, YES, THROTTLE, or ALWAYS.

      • explanation string Required
    • can_rebalance_to_other_node string

      Values are yes, no, worse_balance, throttled, awaiting_info, allocation_delayed, no_valid_shard_copy, or no_attempt.

    • can_remain_decisions array[object]
      Hide can_remain_decisions attributes Show can_remain_decisions attributes object
      • decider string Required
      • decision string Required

        Values are NO, YES, THROTTLE, or ALWAYS.

      • explanation string Required
    • can_remain_on_current_node string

      Values are yes, no, worse_balance, throttled, awaiting_info, allocation_delayed, no_valid_shard_copy, or no_attempt.

    • cluster_info object
      Hide cluster_info attributes Show cluster_info attributes object
      • nodes object Required
        Hide nodes attribute Show nodes attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • node_name string Required
          • least_available object Required
            Hide least_available attributes Show least_available attributes object
            • path string Required
            • total_bytes number Required
            • used_bytes number Required
            • free_bytes number Required
            • free_disk_percent number Required
            • used_disk_percent number Required
          • most_available object Required
            Hide most_available attributes Show most_available attributes object
            • path string Required
            • total_bytes number Required
            • used_bytes number Required
            • free_bytes number Required
            • free_disk_percent number Required
            • used_disk_percent number Required
      • shard_sizes object Required
        Hide shard_sizes attribute Show shard_sizes attribute object
        • * number Additional properties
      • shard_data_set_sizes object
        Hide shard_data_set_sizes attribute Show shard_data_set_sizes attribute object
        • * string Additional properties
      • shard_paths object Required
        Hide shard_paths attribute Show shard_paths attribute object
        • * string Additional properties
      • reserved_sizes array[object] Required
        Hide reserved_sizes attributes Show reserved_sizes attributes object
        • node_id string Required
        • path string Required
        • total number Required
        • shards array[string] Required
    • configured_delay string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • configured_delay_in_millis number

      Time unit for milliseconds

    • current_node object
      Hide current_node attributes Show current_node attributes object
      • id string Required
      • name string Required
      • roles array[string] Required

        Values are master, data, data_cold, data_content, data_frozen, data_hot, data_warm, client, ingest, ml, voting_only, transform, remote_cluster_client, or coordinating_only.

      • attributes object Required
        Hide attributes attribute Show attributes attribute object
        • * string Additional properties
      • transport_address string Required
      • weight_ranking number Required
    • current_state string Required
    • index string Required
    • move_explanation string
    • node_allocation_decisions array[object]
      Hide node_allocation_decisions attributes Show node_allocation_decisions attributes object
      • deciders array[object] Required
        Hide deciders attributes Show deciders attributes object
        • decider string Required
        • decision string Required

          Values are NO, YES, THROTTLE, or ALWAYS.

        • explanation string Required
      • node_attributes object Required
        Hide node_attributes attribute Show node_attributes attribute object
        • * string Additional properties
      • node_decision string Required

        Values are yes, no, worse_balance, throttled, awaiting_info, allocation_delayed, no_valid_shard_copy, or no_attempt.

      • node_id string Required
      • node_name string Required
      • roles array[string] Required

        Values are master, data, data_cold, data_content, data_frozen, data_hot, data_warm, client, ingest, ml, voting_only, transform, remote_cluster_client, or coordinating_only.

      • store object
        Hide store attributes Show store attributes object
        • allocation_id string Required
        • found boolean Required
        • in_sync boolean Required
        • matching_size_in_bytes number Required
        • matching_sync_id boolean Required
        • store_exception string Required
      • transport_address string Required
      • weight_ranking number Required
    • primary boolean Required
    • rebalance_explanation string
    • remaining_delay string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • remaining_delay_in_millis number

      Time unit for milliseconds

    • shard number Required
    • unassigned_info object
      Hide unassigned_info attributes Show unassigned_info attributes object
      • at string | number Required

        A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

        One of:
      • last_allocation_status string
      • reason string Required

        Values are INDEX_CREATED, CLUSTER_RECOVERED, INDEX_REOPENED, DANGLING_INDEX_IMPORTED, NEW_INDEX_RESTORED, EXISTING_INDEX_RESTORED, REPLICA_ADDED, ALLOCATION_FAILED, NODE_LEFT, REROUTE_CANCELLED, REINITIALIZED, REALLOCATED_REPLICA, PRIMARY_FAILED, FORCED_EMPTY_PRIMARY, or MANUAL_ALLOCATION.

      • details string
      • failed_allocation_attempts number
      • delayed boolean
      • allocation_status string
    • note string Generally available; Added in 7.14.0
GET /_cluster/allocation/explain
curl \
 --request GET 'http://api.example.com/_cluster/allocation/explain' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"index\": \"my-index-000001\",\n  \"shard\": 0,\n  \"primary\": false,\n  \"current_node\": \"my-node\"\n}"'
Request example
Run `GET _cluster/allocation/explain` to get an explanation for a shard's current allocation.
{
  "index": "my-index-000001",
  "shard": 0,
  "primary": false,
  "current_node": "my-node"
}
Response examples (200)
An example of an allocation explanation for an unassigned primary shard. In this example, a newly created index has an index setting that requires that it only be allocated to a node named `nonexistent_node`, which does not exist, so the index is unable to allocate.
{
  "index" : "my-index-000001",
  "shard" : 0,
  "primary" : true,
  "current_state" : "unassigned",
  "unassigned_info" : {
    "reason" : "INDEX_CREATED",
    "at" : "2017-01-04T18:08:16.600Z",
    "last_allocation_status" : "no"
  },
  "can_allocate" : "no",
  "allocate_explanation" : "Elasticsearch isn't allowed to allocate this shard to any of the nodes in the cluster. Choose a node to which you expect this shard to be allocated, find this node in the node-by-node explanation, and address the reasons which prevent Elasticsearch from allocating this shard there.",
  "node_allocation_decisions" : [
    {
      "node_id" : "8qt2rY-pT6KNZB3-hGfLnw",
      "node_name" : "node-0",
      "transport_address" : "127.0.0.1:9401",
      "roles" : ["data", "data_cold", "data_content", "data_frozen", "data_hot", "data_warm", "ingest", "master", "ml", "remote_cluster_client", "transform"],
      "node_attributes" : {},
      "node_decision" : "no",
      "weight_ranking" : 1,
      "deciders" : [
        {
          "decider" : "filter",
          "decision" : "NO",
          "explanation" : "node does not match index setting [index.routing.allocation.include] filters [_name:\"nonexistent_node\"]"
        }
      ]
    }
  ]
}
An example of an allocation explanation for an unassigned primary shard that has reached the maximum number of allocation retry attempts. After the maximum number of retries is reached, Elasticsearch stops attempting to allocate the shard in order to prevent infinite retries which may impact cluster performance.
{
  "index" : "my-index-000001",
  "shard" : 0,
  "primary" : true,
  "current_state" : "unassigned",
  "unassigned_info" : {
    "at" : "2017-01-04T18:03:28.464Z",
    "failed shard on node [mEKjwwzLT1yJVb8UxT6anw]: failed recovery, failure RecoveryFailedException",
    "reason": "ALLOCATION_FAILED",
    "failed_allocation_attempts": 5,
    "last_allocation_status": "no",
  },
  "can_allocate": "no",
  "allocate_explanation": "cannot allocate because allocation is not permitted to any of the nodes",
  "node_allocation_decisions" : [
    {
      "node_id" : "3sULLVJrRneSg0EfBB-2Ew",
      "node_name" : "node_t0",
      "transport_address" : "127.0.0.1:9400",
      "roles" : ["data_content", "data_hot"],
      "node_decision" : "no",
      "store" : {
        "matching_size" : "4.2kb",
        "matching_size_in_bytes" : 4325
      },
      "deciders" : [
        {
          "decider": "max_retry",
          "decision" : "NO",
          "explanation": "shard has exceeded the maximum number of retries [5] on failed allocation attempts - manually call [POST /_cluster/reroute?retry_failed] to retry, [unassigned_info[[reason=ALLOCATION_FAILED], at[2024-07-30T21:04:12.166Z], failed_attempts[5], failed_nodes[[mEKjwwzLT1yJVb8UxT6anw]], delayed=false, details[failed shard on node [mEKjwwzLT1yJVb8UxT6anw]: failed recovery, failure RecoveryFailedException], allocation_status[deciders_no]]]"
        }
      ]
    }
  ]
}








Clear cluster voting config exclusions Generally available; Added in 7.0.0

DELETE /_cluster/voting_config_exclusions

Remove master-eligible nodes from the voting configuration exclusion list.

External documentation

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

  • wait_for_removal boolean

    Specifies whether to wait for all excluded nodes to be removed from the cluster before clearing the voting configuration exclusions list. Defaults to true, meaning that all excluded nodes must be removed from the cluster before this API takes any action. If set to false then the voting configuration exclusions list is cleared even if some excluded nodes are still in the cluster.

Responses

  • 200 application/json
DELETE /_cluster/voting_config_exclusions
curl \
 --request DELETE 'http://api.example.com/_cluster/voting_config_exclusions' \
 --header "Authorization: $API_KEY"
















Get cluster info Generally available; Added in 8.9.0

GET /_info/{target}

Returns basic information about the cluster.

Path parameters

  • target string | array[string] Required

    Limits the information returned to the specific target. Supports a comma-separated list, such as http,ingest.

    Values are _all, http, ingest, thread_pool, or script.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • cluster_name string Required
    • http object
      Hide http attributes Show http attributes object
      • current_open number

        Current number of open HTTP connections for the node.

      • total_opened number

        Total number of HTTP connections opened for the node.

      • clients array[object]

        Information on current and recently-closed HTTP client connections. Clients that have been closed longer than the http.client_stats.closed_channels.max_age setting will not be represented here.

        Hide clients attributes Show clients attributes object
        • id number

          Unique ID for the HTTP client.

        • agent string

          Reported agent for the HTTP client. If unavailable, this property is not included in the response.

        • local_address string

          Local address for the HTTP connection.

        • remote_address string

          Remote address for the HTTP connection.

        • last_uri string

          The URI of the client’s most recent request.

        • opened_time_millis number

          Time at which the client opened the connection.

        • closed_time_millis number

          Time at which the client closed the connection if the connection is closed.

        • last_request_time_millis number

          Time of the most recent request from this client.

        • request_count number

          Number of requests from this client.

        • request_size_bytes number

          Cumulative size in bytes of all requests from this client.

        • x_opaque_id string

          Value from the client’s x-opaque-id HTTP header. If unavailable, this property is not included in the response.

      • routes object Required Generally available; Added in 8.12.0

        Detailed HTTP stats broken down by route

        Hide routes attribute Show routes attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • requests object Required
            Hide requests attributes Show requests attributes object
            • count number Required
            • total_size_in_bytes number Required
            • size_histogram array[object] Required
          • responses object Required
            Hide responses attributes Show responses attributes object
            • count number Required
            • total_size_in_bytes number Required
            • handling_time_histogram array[object] Required
            • size_histogram array[object] Required
    • ingest object
      Hide ingest attributes Show ingest attributes object
      • pipelines object

        Contains statistics about ingest pipelines for the node.

        Hide pipelines attribute Show pipelines attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • count number Required

            Total number of documents ingested during the lifetime of this node.

          • current number Required

            Total number of documents currently being ingested.

          • failed number Required

            Total number of failed ingest operations during the lifetime of this node.

          • processors array[object] Required

            Total number of ingest processors.

            Hide processors attribute Show processors attribute object
            • * object Additional properties
          • time_in_millis number

            Time unit for milliseconds

          • ingested_as_first_pipeline_in_bytes number Required Generally available; Added in 8.15.0

            Total number of bytes of all documents ingested by the pipeline. This field is only present on pipelines which are the first to process a document. Thus, it is not present on pipelines which only serve as a final pipeline after a default pipeline, a pipeline run after a reroute processor, or pipelines in pipeline processors.

          • produced_as_first_pipeline_in_bytes number Required Generally available; Added in 8.15.0

            Total number of bytes of all documents produced by the pipeline. This field is only present on pipelines which are the first to process a document. Thus, it is not present on pipelines which only serve as a final pipeline after a default pipeline, a pipeline run after a reroute processor, or pipelines in pipeline processors. In situations where there are subsequent pipelines, the value represents the size of the document after all pipelines have run.

      • total object
        Hide total attributes Show total attributes object
        • count number Required

          Total number of documents ingested during the lifetime of this node.

        • current number Required

          Total number of documents currently being ingested.

        • failed number Required

          Total number of failed ingest operations during the lifetime of this node.

        • time_in_millis number

          Time unit for milliseconds

    • thread_pool object
      Hide thread_pool attribute Show thread_pool attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • active number

          Number of active threads in the thread pool.

        • completed number

          Number of tasks completed by the thread pool executor.

        • largest number

          Highest number of active threads in the thread pool.

        • queue number

          Number of tasks in queue for the thread pool.

        • rejected number

          Number of tasks rejected by the thread pool executor.

        • threads number

          Number of threads in the thread pool.

    • script object
      Hide script attributes Show script attributes object
      • cache_evictions number

        Total number of times the script cache has evicted old data.

      • compilations number

        Total number of inline script compilations performed by the node.

      • compilations_history object

        Contains this recent history of script compilations.

        Hide compilations_history attribute Show compilations_history attribute object
        • * number Additional properties
      • compilation_limit_triggered number

        Total number of times the script compilation circuit breaker has limited inline script compilations.

      • contexts array[object]
        Hide contexts attributes Show contexts attributes object
        • context string
        • compilations number
        • cache_evictions number
        • compilation_limit_triggered number
GET /_info/{target}
curl \
 --request GET 'http://api.example.com/_info/{target}' \
 --header "Authorization: $API_KEY"

Get the pending cluster tasks Generally available

GET /_cluster/pending_tasks

Get information about cluster-level changes (such as create index, update mapping, allocate or fail shard) that have not yet taken effect.

NOTE: This API returns a list of any pending updates to the cluster state. These are distinct from the tasks reported by the task management API which include periodic tasks and tasks initiated by the user, such as node stats, search queries, or create index requests. However, if a user-initiated task such as a create index command causes a cluster state update, the activity of this task might be reported by both task api and pending cluster tasks API.

Required authorization

  • Cluster privileges: monitor

Query parameters

  • local boolean

    If true, the request retrieves information from the local node only. If false, information is retrieved from the master node.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • tasks array[object] Required
      Hide tasks attributes Show tasks attributes object
      • executing boolean Required

        Indicates whether the pending tasks are currently executing or not.

      • insert_order number Required

        The number that represents when the task has been inserted into the task queue.

      • priority string Required

        The priority of the pending task. The valid priorities in descending priority order are: IMMEDIATE > URGENT > HIGH > NORMAL > LOW > LANGUID.

      • source string Required

        A general description of the cluster task that may include a reason and origin.

      • time_in_queue string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • time_in_queue_millis number

        Time unit for milliseconds

GET /_cluster/pending_tasks
curl \
 --request GET 'http://api.example.com/_cluster/pending_tasks' \
 --header "Authorization: $API_KEY"

Get remote cluster information Generally available; Added in 6.1.0

GET /_remote/info

Get information about configured remote clusters. The API returns connection and endpoint information keyed by the configured remote cluster alias.


This API returns information that reflects current state on the local cluster. The connected field does not necessarily reflect whether a remote cluster is down or unavailable, only whether there is currently an open connection to it. Elasticsearch does not spontaneously try to reconnect to a disconnected remote cluster. To trigger a reconnection, attempt a cross-cluster search, ES|QL cross-cluster search, or try the /_resolve/cluster endpoint.

Required authorization

  • Cluster privileges: monitor
External documentation

Responses

  • 200 application/json
GET /_remote/info
curl \
 --request GET 'http://api.example.com/_remote/info' \
 --header "Authorization: $API_KEY"












Get the cluster state Generally available; Added in 1.3.0

GET /_cluster/state/{metric}/{index}

Get comprehensive information about the state of the cluster.

The cluster state is an internal data structure which keeps track of a variety of information needed by every node, including the identity and attributes of the other nodes in the cluster; cluster-wide settings; index metadata, including the mapping and settings for each index; the location and status of every shard copy in the cluster.

The elected master node ensures that every node in the cluster has a copy of the same cluster state. This API lets you retrieve a representation of this internal state for debugging or diagnostic purposes. You may need to consult the Elasticsearch source code to determine the precise meaning of the response.

By default the API will route requests to the elected master node since this node is the authoritative source of cluster states. You can also retrieve the cluster state held on the node handling the API request by adding the ?local=true query parameter.

Elasticsearch may need to expend significant effort to compute a response to this API in larger clusters, and the response may comprise a very large quantity of data. If you use this API repeatedly, your cluster may become unstable.

WARNING: The response is a representation of an internal data structure. Its format is not subject to the same compatibility guarantees as other more stable APIs and may change from version to version. Do not query this API using external monitoring tools. Instead, obtain the information you require using other more stable cluster APIs.

Required authorization

  • Cluster privileges: monitor,manage

Path parameters

  • metric string | array[string] Required

    Limit the information returned to the specified metrics

  • index string | array[string] Required

    A comma-separated list of index names; use _all or empty string to perform the operation on all indices

Query parameters

  • allow_no_indices boolean

    Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)

  • expand_wildcards string | array[string]

    Whether to expand wildcard expression to concrete indices that are open, closed or both.

    Values are all, open, closed, hidden, or none.

  • flat_settings boolean

    Return settings in flat format (default: false)

  • ignore_unavailable boolean

    Whether specified concrete indices should be ignored when unavailable (missing or closed)

  • local boolean

    Return local information, do not retrieve the state from master node (default: false)

  • master_timeout string

    Specify timeout for connection to master

    Values are -1 or 0.

  • wait_for_metadata_version number

    Wait for the metadata version to be equal or greater than the specified metadata version

  • wait_for_timeout string

    The maximum time to wait for wait_for_metadata_version before timing out

    Values are -1 or 0.

Responses

  • 200 application/json
GET /_cluster/state/{metric}/{index}
curl \
 --request GET 'http://api.example.com/_cluster/state/{metric}/{index}' \
 --header "Authorization: $API_KEY"

Get cluster statistics Generally available; Added in 1.3.0

GET /_cluster/stats

Get basic index metrics (shard numbers, store size, memory usage) and information about the current nodes that form the cluster (number, roles, os, jvm versions, memory usage, cpu and installed plugins).

Required authorization

  • Cluster privileges: monitor

Query parameters

  • include_remotes boolean

    Include remote cluster data into the response

  • timeout string

    Period to wait for each node to respond. If a node does not respond before its timeout expires, the response does not include its stats. However, timed out nodes are included in the response’s _nodes.failed property. Defaults to no timeout.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object

      Contains statistics about the number of nodes selected by the request.

      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide failures attributes Show failures attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • cluster_uuid string Required
    • indices object Required
      Hide indices attributes Show indices attributes object
      • analysis object Required
        Hide analysis attributes Show analysis attributes object
        • analyzer_types array[object] Required

          Contains statistics about analyzer types used in selected nodes.

          Hide analyzer_types attributes Show analyzer_types attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

        • built_in_analyzers array[object] Required

          Contains statistics about built-in analyzers used in selected nodes.

          Hide built_in_analyzers attributes Show built_in_analyzers attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

        • built_in_char_filters array[object] Required

          Contains statistics about built-in character filters used in selected nodes.

          Hide built_in_char_filters attributes Show built_in_char_filters attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

        • built_in_filters array[object] Required

          Contains statistics about built-in token filters used in selected nodes.

          Hide built_in_filters attributes Show built_in_filters attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

        • built_in_tokenizers array[object] Required

          Contains statistics about built-in tokenizers used in selected nodes.

          Hide built_in_tokenizers attributes Show built_in_tokenizers attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

        • char_filter_types array[object] Required

          Contains statistics about character filter types used in selected nodes.

          Hide char_filter_types attributes Show char_filter_types attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

        • filter_types array[object] Required

          Contains statistics about token filter types used in selected nodes.

          Hide filter_types attributes Show filter_types attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

        • tokenizer_types array[object] Required

          Contains statistics about tokenizer types used in selected nodes.

          Hide tokenizer_types attributes Show tokenizer_types attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

      • completion object Required
        Hide completion attributes Show completion attributes object
        • size_in_bytes number Required

          Total amount, in bytes, of memory used for completion across all shards assigned to selected nodes.

        • size number | string

        • fields object
          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • size
            • size_in_bytes number Required
      • count number Required

        Total number of indices with shards assigned to selected nodes.

      • docs object Required
        Hide docs attributes Show docs attributes object
        • count number Required

          Total number of non-deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments and may include documents from nested fields.

        • deleted number

          Total number of deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments. Elasticsearch reclaims the disk space of deleted Lucene documents when a segment is merged.

      • fielddata object Required
        Hide fielddata attributes Show fielddata attributes object
        • evictions number
        • memory_size number | string

        • memory_size_in_bytes number Required
        • fields object
          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • memory_size
            • memory_size_in_bytes number Required
      • query_cache object Required
        Hide query_cache attributes Show query_cache attributes object
        • cache_count number Required

          Total number of entries added to the query cache across all shards assigned to selected nodes. This number includes current and evicted entries.

        • cache_size number Required

          Total number of entries currently in the query cache across all shards assigned to selected nodes.

        • evictions number Required

          Total number of query cache evictions across all shards assigned to selected nodes.

        • hit_count number Required

          Total count of query cache hits across all shards assigned to selected nodes.

        • memory_size number | string

        • memory_size_in_bytes number Required

          Total amount, in bytes, of memory used for the query cache across all shards assigned to selected nodes.

        • miss_count number Required

          Total count of query cache misses across all shards assigned to selected nodes.

        • total_count number Required

          Total count of hits and misses in the query cache across all shards assigned to selected nodes.

      • segments object Required
        Hide segments attributes Show segments attributes object
        • count number Required

          Total number of segments across all shards assigned to selected nodes.

        • doc_values_memory number | string

        • doc_values_memory_in_bytes number Required

          Total amount, in bytes, of memory used for doc values across all shards assigned to selected nodes.

        • file_sizes object Required

          This object is not populated by the cluster stats API. To get information on segment files, use the node stats API.

          Hide file_sizes attribute Show file_sizes attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • description string Required
            • size_in_bytes number Required
            • min_size_in_bytes number
            • max_size_in_bytes number
            • average_size_in_bytes number
            • count number
        • fixed_bit_set number | string

        • fixed_bit_set_memory_in_bytes number Required

          Total amount of memory, in bytes, used by fixed bit sets across all shards assigned to selected nodes.

        • index_writer_memory number | string

        • index_writer_max_memory_in_bytes number
        • index_writer_memory_in_bytes number Required

          Total amount, in bytes, of memory used by all index writers across all shards assigned to selected nodes.

        • max_unsafe_auto_id_timestamp number Required

          Unix timestamp, in milliseconds, of the most recently retried indexing request.

        • memory number | string

        • memory_in_bytes number Required

          Total amount, in bytes, of memory used for segments across all shards assigned to selected nodes.

        • norms_memory number | string

        • norms_memory_in_bytes number Required

          Total amount, in bytes, of memory used for normalization factors across all shards assigned to selected nodes.

        • points_memory number | string

        • points_memory_in_bytes number Required

          Total amount, in bytes, of memory used for points across all shards assigned to selected nodes.

        • stored_memory number | string

        • stored_fields_memory_in_bytes number Required

          Total amount, in bytes, of memory used for stored fields across all shards assigned to selected nodes.

        • terms_memory_in_bytes number Required

          Total amount, in bytes, of memory used for terms across all shards assigned to selected nodes.

        • terms_memory number | string

        • term_vectory_memory number | string

        • term_vectors_memory_in_bytes number Required

          Total amount, in bytes, of memory used for term vectors across all shards assigned to selected nodes.

        • version_map_memory number | string

        • version_map_memory_in_bytes number Required

          Total amount, in bytes, of memory used by all version maps across all shards assigned to selected nodes.

      • shards object Required

        Contains statistics about shards assigned to selected nodes.

        Hide shards attributes Show shards attributes object
        • index object
          Hide index attributes Show index attributes object
          • primaries object Required
            Hide primaries attributes Show primaries attributes object
            • avg number Required

              Mean number of shards in an index, counting only shards assigned to selected nodes.

            • max number Required

              Maximum number of shards in an index, counting only shards assigned to selected nodes.

            • min number Required

              Minimum number of shards in an index, counting only shards assigned to selected nodes.

          • replication object Required
            Hide replication attributes Show replication attributes object
            • avg number Required

              Mean number of shards in an index, counting only shards assigned to selected nodes.

            • max number Required

              Maximum number of shards in an index, counting only shards assigned to selected nodes.

            • min number Required

              Minimum number of shards in an index, counting only shards assigned to selected nodes.

          • shards object Required
            Hide shards attributes Show shards attributes object
            • avg number Required

              Mean number of shards in an index, counting only shards assigned to selected nodes.

            • max number Required

              Maximum number of shards in an index, counting only shards assigned to selected nodes.

            • min number Required

              Minimum number of shards in an index, counting only shards assigned to selected nodes.

        • primaries number

          Number of primary shards assigned to selected nodes.

        • replication number

          Ratio of replica shards to primary shards across all selected nodes.

        • total number

          Total number of shards assigned to selected nodes.

      • store object Required
        Hide store attributes Show store attributes object
        • size number | string

        • size_in_bytes number Required

          Total size, in bytes, of all shards assigned to selected nodes.

        • reserved number | string

        • reserved_in_bytes number Required

          A prediction, in bytes, of how much larger the shard stores will eventually grow due to ongoing peer recoveries, restoring snapshots, and similar activities.

        • total_data_set_size number | string

        • total_data_set_size_in_bytes number

          Total data set size, in bytes, of all shards assigned to selected nodes. This includes the size of shards not stored fully on the nodes, such as the cache for partially mounted indices.

      • mappings object Required
        Hide mappings attributes Show mappings attributes object
        • field_types array[object] Required

          Contains statistics about field data types used in selected nodes.

          Hide field_types attributes Show field_types attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

        • runtime_field_types array[object]

          Contains statistics about runtime field data types used in selected nodes.

          Hide runtime_field_types attributes Show runtime_field_types attributes object
          • chars_max number Required

            Maximum number of characters for a single runtime field script.

          • chars_total number Required

            Total number of characters for the scripts that define the current runtime field data type.

          • count number Required

            Number of runtime fields mapped to the field data type in selected nodes.

          • doc_max number Required

            Maximum number of accesses to doc_values for a single runtime field script

          • doc_total number Required

            Total number of accesses to doc_values for the scripts that define the current runtime field data type.

          • index_count number Required

            Number of indices containing a mapping of the runtime field data type in selected nodes.

          • lang array[string] Required

            Script languages used for the runtime fields scripts.

          • lines_max number Required

            Maximum number of lines for a single runtime field script.

          • lines_total number Required

            Total number of lines for the scripts that define the current runtime field data type.

          • name string Required
          • scriptless_count number Required

            Number of runtime fields that don’t declare a script.

          • shadowed_count number Required

            Number of runtime fields that shadow an indexed field.

          • source_max number Required

            Maximum number of accesses to _source for a single runtime field script.

          • source_total number Required

            Total number of accesses to _source for the scripts that define the current runtime field data type.

        • total_field_count number

          Total number of fields in all non-system indices.

        • total_deduplicated_field_count number

          Total number of fields in all non-system indices, accounting for mapping deduplication.

        • total_deduplicated_mapping_size number | string

        • total_deduplicated_mapping_size_in_bytes number

          Total size of all mappings, in bytes, after deduplication and compression.

      • versions array[object]

        Contains statistics about analyzers and analyzer components used in selected nodes.

        Hide versions attributes Show versions attributes object
        • index_count number Required
        • primary_shard_count number Required
        • total_primary_bytes number Required
        • version string Required
    • nodes object Required
      Hide nodes attributes Show nodes attributes object
      • count object Required
        Hide count attributes Show count attributes object
        • coordinating_only number Required
        • data number Required
        • data_cold number Required
        • data_content number Required
        • data_frozen number Generally available; Added in 7.13.0
        • data_hot number Required
        • data_warm number Required
        • ingest number Required
        • master number Required
        • ml number Required
        • remote_cluster_client number Required
        • total number Required
        • transform number Required
        • voting_only number Required
      • discovery_types object Required

        Contains statistics about the discovery types used by selected nodes.

        Hide discovery_types attribute Show discovery_types attribute object
        • * number Additional properties
      • fs object Required
        Hide fs attributes Show fs attributes object
        • available_in_bytes number Required

          Total number of bytes available to JVM in file stores across all selected nodes. Depending on operating system or process-level restrictions, this number may be less than nodes.fs.free_in_byes. This is the actual amount of free disk space the selected Elasticsearch nodes can use.

        • free_in_bytes number Required

          Total number of unallocated bytes in file stores across all selected nodes.

        • total_in_bytes number Required

          Total size, in bytes, of all file stores across all selected nodes.

      • indexing_pressure object Required
        Hide indexing_pressure attribute Show indexing_pressure attribute object
        • memory object Required
          Hide memory attributes Show memory attributes object
          • current object Required
            Hide current attributes Show current attributes object
            • all_in_bytes number Required
            • combined_coordinating_and_primary_in_bytes number Required
            • coordinating_in_bytes number Required
            • coordinating_rejections number
            • primary_in_bytes number Required
            • primary_rejections number
            • replica_in_bytes number Required
            • replica_rejections number
          • limit_in_bytes number Required
          • total object Required
            Hide total attributes Show total attributes object
            • all_in_bytes number Required
            • combined_coordinating_and_primary_in_bytes number Required
            • coordinating_in_bytes number Required
            • coordinating_rejections number
            • primary_in_bytes number Required
            • primary_rejections number
            • replica_in_bytes number Required
            • replica_rejections number
      • ingest object Required
        Hide ingest attributes Show ingest attributes object
        • number_of_pipelines number Required
        • processor_stats object Required
          Hide processor_stats attribute Show processor_stats attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • count number Required
            • current number Required
            • failed number Required
            • time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • jvm object Required
        Hide jvm attributes Show jvm attributes object
        • Time unit for milliseconds

        • mem object Required
          Hide mem attributes Show mem attributes object
          • heap_max_in_bytes number Required

            Maximum amount of memory, in bytes, available for use by the heap across all selected nodes.

          • heap_used_in_bytes number Required

            Memory, in bytes, currently in use by the heap across all selected nodes.

        • threads number Required

          Number of active threads in use by JVM across all selected nodes.

        • versions array[object] Required

          Contains statistics about the JVM versions used by selected nodes.

          Hide versions attributes Show versions attributes object
          • bundled_jdk boolean Required

            Always true. All distributions come with a bundled Java Development Kit (JDK).

          • count number Required

            Total number of selected nodes using JVM.

          • using_bundled_jdk boolean Required

            If true, a bundled JDK is in use by JVM.

          • version string Required
          • vm_name string Required

            Name of the JVM.

          • vm_vendor string Required

            Vendor of the JVM.

          • vm_version string Required
      • network_types object Required
        Hide network_types attributes Show network_types attributes object
        • http_types object Required

          Contains statistics about the HTTP network types used by selected nodes.

          Hide http_types attribute Show http_types attribute object
          • * number Additional properties
        • transport_types object Required

          Contains statistics about the transport network types used by selected nodes.

          Hide transport_types attribute Show transport_types attribute object
          • * number Additional properties
      • os object Required
        Hide os attributes Show os attributes object
        • allocated_processors number Required

          Number of processors used to calculate thread pool size across all selected nodes. This number can be set with the processors setting of a node and defaults to the number of processors reported by the operating system. In both cases, this number will never be larger than 32.

        • architectures array[object]

          Contains statistics about processor architectures (for example, x86_64 or aarch64) used by selected nodes.

          Hide architectures attributes Show architectures attributes object
          • arch string Required

            Name of an architecture used by one or more selected nodes.

          • count number Required

            Number of selected nodes using the architecture.

        • available_processors number Required

          Number of processors available to JVM across all selected nodes.

        • mem object Required
          Hide mem attributes Show mem attributes object
          • adjusted_total_in_bytes number Generally available; Added in 7.16.0

            Total amount, in bytes, of memory across all selected nodes, but using the value specified using the es.total_memory_bytes system property instead of measured total memory for those nodes where that system property was set.

          • free_in_bytes number Required

            Amount, in bytes, of free physical memory across all selected nodes.

          • free_percent number Required

            Percentage of free physical memory across all selected nodes.

          • total_in_bytes number Required

            Total amount, in bytes, of physical memory across all selected nodes.

          • used_in_bytes number Required

            Amount, in bytes, of physical memory in use across all selected nodes.

          • used_percent number Required

            Percentage of physical memory in use across all selected nodes.

        • names array[object] Required

          Contains statistics about operating systems used by selected nodes.

          Hide names attributes Show names attributes object
          • count number Required

            Number of selected nodes using the operating system.

          • name string Required
        • pretty_names array[object] Required

          Contains statistics about operating systems used by selected nodes.

          Hide pretty_names attributes Show pretty_names attributes object
          • count number Required

            Number of selected nodes using the operating system.

          • pretty_name string Required
      • packaging_types array[object] Required

        Contains statistics about Elasticsearch distributions installed on selected nodes.

        Hide packaging_types attributes Show packaging_types attributes object
        • count number Required

          Number of selected nodes using the distribution flavor and file type.

        • flavor string Required

          Type of Elasticsearch distribution. This is always default.

        • type string Required

          File type (such as tar or zip) used for the distribution package.

      • plugins array[object] Required

        Contains statistics about installed plugins and modules by selected nodes. If no plugins or modules are installed, this array is empty.

        Hide plugins attributes Show plugins attributes object
        • classname string Required
        • description string Required
        • elasticsearch_version string Required
        • extended_plugins array[string] Required
        • has_native_controller boolean Required
        • java_version string Required
        • name string Required
        • version string Required
        • licensed boolean Required
      • process object Required
        Hide process attributes Show process attributes object
        • cpu object Required
          Hide cpu attribute Show cpu attribute object
          • percent number Required

            Percentage of CPU used across all selected nodes. Returns -1 if not supported.

        • open_file_descriptors object Required
          Hide open_file_descriptors attributes Show open_file_descriptors attributes object
          • avg number Required

            Average number of concurrently open file descriptors. Returns -1 if not supported.

          • max number Required

            Maximum number of concurrently open file descriptors allowed across all selected nodes. Returns -1 if not supported.

          • min number Required

            Minimum number of concurrently open file descriptors across all selected nodes. Returns -1 if not supported.

      • versions array[string] Required

        Array of Elasticsearch versions used on selected nodes.

    • status string Required

      Values are green, GREEN, yellow, YELLOW, red, or RED.

    • timestamp number Required

      Unix timestamp, in milliseconds, for the last time the cluster statistics were refreshed.

    • ccs object Required
      Hide ccs attributes Show ccs attributes object
      • clusters object

        Contains remote cluster settings and metrics collected from them. The keys are cluster names, and the values are per-cluster data. Only present if include_remotes option is set to true.

        Hide clusters attribute Show clusters attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • cluster_uuid string Required

            The UUID of the remote cluster.

          • mode string Required

            The connection mode used to communicate with the remote cluster.

          • skip_unavailable boolean Required

            The skip_unavailable setting used for this remote cluster.

          • transport_compress string Required

            Transport compression setting used for this remote cluster.

          • status string Required

            Values are green, GREEN, yellow, YELLOW, red, or RED.

          • version array[string] Required

            The list of Elasticsearch versions used by the nodes on the remote cluster.

          • nodes_count number Required

            The total count of nodes in the remote cluster.

          • shards_count number Required

            The total number of shards in the remote cluster.

          • indices_count number Required

            The total number of indices in the remote cluster.

          • indices_total_size_in_bytes number Required

            Total data set size, in bytes, of all shards assigned to selected nodes.

          • indices_total_size string

            Total data set size of all shards assigned to selected nodes, as a human-readable string.

          • max_heap_in_bytes number Required

            Maximum amount of memory, in bytes, available for use by the heap across the nodes of the remote cluster.

          • max_heap string

            Maximum amount of memory available for use by the heap across the nodes of the remote cluster, as a human-readable string.

          • mem_total_in_bytes number Required

            Total amount, in bytes, of physical memory across the nodes of the remote cluster.

          • mem_total string

            Total amount of physical memory across the nodes of the remote cluster, as a human-readable string.

      • _esql object
        Hide _esql attributes Show _esql attributes object
        • total number Required

          The total number of cross-cluster search requests that have been executed by the cluster.

        • success number Required

          The total number of cross-cluster search requests that have been successfully executed by the cluster.

        • skipped number Required

          The total number of cross-cluster search requests (successful or failed) that had at least one remote cluster skipped.

        • took object Required
          Hide took attributes Show took attributes object
          • Time unit for milliseconds

          • Time unit for milliseconds

          • Time unit for milliseconds

        • took_mrt_true object
          Hide took_mrt_true attributes Show took_mrt_true attributes object
          • Time unit for milliseconds

          • Time unit for milliseconds

          • Time unit for milliseconds

        • took_mrt_false object
          Hide took_mrt_false attributes Show took_mrt_false attributes object
          • Time unit for milliseconds

          • Time unit for milliseconds

          • Time unit for milliseconds

        • remotes_per_search_max number Required

          The maximum number of remote clusters that were queried in a single cross-cluster search request.

        • remotes_per_search_avg number Required

          The average number of remote clusters that were queried in a single cross-cluster search request.

        • failure_reasons object Required

          Statistics about the reasons for cross-cluster search request failures. The keys are the failure reason names and the values are the number of requests that failed for that reason.

          Hide failure_reasons attribute Show failure_reasons attribute object
          • * number Additional properties
        • features object Required

          The keys are the names of the search feature, and the values are the number of requests that used that feature. Single request can use more than one feature (e.g. both async and wildcard).

          Hide features attribute Show features attribute object
          • * number Additional properties
        • clients object Required

          Statistics about the clients that executed cross-cluster search requests. The keys are the names of the clients, and the values are the number of requests that were executed by that client. Only known clients (such as kibana or elasticsearch) are counted.

          Hide clients attribute Show clients attribute object
          • * number Additional properties
        • clusters object Required

          Statistics about the clusters that were queried in cross-cluster search requests. The keys are cluster names, and the values are per-cluster telemetry data. This also includes the local cluster itself, which uses the name (local).

          Hide clusters attribute Show clusters attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • total number Required

              The total number of successful (not skipped) cross-cluster search requests that were executed against this cluster. This may include requests where partial results were returned, but not requests in which the cluster has been skipped entirely.

            • skipped number Required

              The total number of cross-cluster search requests for which this cluster was skipped.

            • took object Required
GET /_cluster/stats
curl \
 --request GET 'http://api.example.com/_cluster/stats' \
 --header "Authorization: $API_KEY"

Get cluster statistics Generally available; Added in 1.3.0

GET /_cluster/stats/nodes/{node_id}

Get basic index metrics (shard numbers, store size, memory usage) and information about the current nodes that form the cluster (number, roles, os, jvm versions, memory usage, cpu and installed plugins).

Required authorization

  • Cluster privileges: monitor

Path parameters

  • node_id string | array[string] Required

    Comma-separated list of node filters used to limit returned information. Defaults to all nodes in the cluster.

Query parameters

  • include_remotes boolean

    Include remote cluster data into the response

  • timeout string

    Period to wait for each node to respond. If a node does not respond before its timeout expires, the response does not include its stats. However, timed out nodes are included in the response’s _nodes.failed property. Defaults to no timeout.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object

      Contains statistics about the number of nodes selected by the request.

      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide failures attributes Show failures attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • cluster_uuid string Required
    • indices object Required
      Hide indices attributes Show indices attributes object
      • analysis object Required
        Hide analysis attributes Show analysis attributes object
        • analyzer_types array[object] Required

          Contains statistics about analyzer types used in selected nodes.

          Hide analyzer_types attributes Show analyzer_types attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

        • built_in_analyzers array[object] Required

          Contains statistics about built-in analyzers used in selected nodes.

          Hide built_in_analyzers attributes Show built_in_analyzers attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

        • built_in_char_filters array[object] Required

          Contains statistics about built-in character filters used in selected nodes.

          Hide built_in_char_filters attributes Show built_in_char_filters attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

        • built_in_filters array[object] Required

          Contains statistics about built-in token filters used in selected nodes.

          Hide built_in_filters attributes Show built_in_filters attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

        • built_in_tokenizers array[object] Required

          Contains statistics about built-in tokenizers used in selected nodes.

          Hide built_in_tokenizers attributes Show built_in_tokenizers attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

        • char_filter_types array[object] Required

          Contains statistics about character filter types used in selected nodes.

          Hide char_filter_types attributes Show char_filter_types attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

        • filter_types array[object] Required

          Contains statistics about token filter types used in selected nodes.

          Hide filter_types attributes Show filter_types attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

        • tokenizer_types array[object] Required

          Contains statistics about tokenizer types used in selected nodes.

          Hide tokenizer_types attributes Show tokenizer_types attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

      • completion object Required
        Hide completion attributes Show completion attributes object
        • size_in_bytes number Required

          Total amount, in bytes, of memory used for completion across all shards assigned to selected nodes.

        • size number | string

        • fields object
          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • size
            • size_in_bytes number Required
      • count number Required

        Total number of indices with shards assigned to selected nodes.

      • docs object Required
        Hide docs attributes Show docs attributes object
        • count number Required

          Total number of non-deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments and may include documents from nested fields.

        • deleted number

          Total number of deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments. Elasticsearch reclaims the disk space of deleted Lucene documents when a segment is merged.

      • fielddata object Required
        Hide fielddata attributes Show fielddata attributes object
        • evictions number
        • memory_size number | string

        • memory_size_in_bytes number Required
        • fields object
          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • memory_size
            • memory_size_in_bytes number Required
      • query_cache object Required
        Hide query_cache attributes Show query_cache attributes object
        • cache_count number Required

          Total number of entries added to the query cache across all shards assigned to selected nodes. This number includes current and evicted entries.

        • cache_size number Required

          Total number of entries currently in the query cache across all shards assigned to selected nodes.

        • evictions number Required

          Total number of query cache evictions across all shards assigned to selected nodes.

        • hit_count number Required

          Total count of query cache hits across all shards assigned to selected nodes.

        • memory_size number | string

        • memory_size_in_bytes number Required

          Total amount, in bytes, of memory used for the query cache across all shards assigned to selected nodes.

        • miss_count number Required

          Total count of query cache misses across all shards assigned to selected nodes.

        • total_count number Required

          Total count of hits and misses in the query cache across all shards assigned to selected nodes.

      • segments object Required
        Hide segments attributes Show segments attributes object
        • count number Required

          Total number of segments across all shards assigned to selected nodes.

        • doc_values_memory number | string

        • doc_values_memory_in_bytes number Required

          Total amount, in bytes, of memory used for doc values across all shards assigned to selected nodes.

        • file_sizes object Required

          This object is not populated by the cluster stats API. To get information on segment files, use the node stats API.

          Hide file_sizes attribute Show file_sizes attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • description string Required
            • size_in_bytes number Required
            • min_size_in_bytes number
            • max_size_in_bytes number
            • average_size_in_bytes number
            • count number
        • fixed_bit_set number | string

        • fixed_bit_set_memory_in_bytes number Required

          Total amount of memory, in bytes, used by fixed bit sets across all shards assigned to selected nodes.

        • index_writer_memory number | string

        • index_writer_max_memory_in_bytes number
        • index_writer_memory_in_bytes number Required

          Total amount, in bytes, of memory used by all index writers across all shards assigned to selected nodes.

        • max_unsafe_auto_id_timestamp number Required

          Unix timestamp, in milliseconds, of the most recently retried indexing request.

        • memory number | string

        • memory_in_bytes number Required

          Total amount, in bytes, of memory used for segments across all shards assigned to selected nodes.

        • norms_memory number | string

        • norms_memory_in_bytes number Required

          Total amount, in bytes, of memory used for normalization factors across all shards assigned to selected nodes.

        • points_memory number | string

        • points_memory_in_bytes number Required

          Total amount, in bytes, of memory used for points across all shards assigned to selected nodes.

        • stored_memory number | string

        • stored_fields_memory_in_bytes number Required

          Total amount, in bytes, of memory used for stored fields across all shards assigned to selected nodes.

        • terms_memory_in_bytes number Required

          Total amount, in bytes, of memory used for terms across all shards assigned to selected nodes.

        • terms_memory number | string

        • term_vectory_memory number | string

        • term_vectors_memory_in_bytes number Required

          Total amount, in bytes, of memory used for term vectors across all shards assigned to selected nodes.

        • version_map_memory number | string

        • version_map_memory_in_bytes number Required

          Total amount, in bytes, of memory used by all version maps across all shards assigned to selected nodes.

      • shards object Required

        Contains statistics about shards assigned to selected nodes.

        Hide shards attributes Show shards attributes object
        • index object
          Hide index attributes Show index attributes object
          • primaries object Required
            Hide primaries attributes Show primaries attributes object
            • avg number Required

              Mean number of shards in an index, counting only shards assigned to selected nodes.

            • max number Required

              Maximum number of shards in an index, counting only shards assigned to selected nodes.

            • min number Required

              Minimum number of shards in an index, counting only shards assigned to selected nodes.

          • replication object Required
            Hide replication attributes Show replication attributes object
            • avg number Required

              Mean number of shards in an index, counting only shards assigned to selected nodes.

            • max number Required

              Maximum number of shards in an index, counting only shards assigned to selected nodes.

            • min number Required

              Minimum number of shards in an index, counting only shards assigned to selected nodes.

          • shards object Required
            Hide shards attributes Show shards attributes object
            • avg number Required

              Mean number of shards in an index, counting only shards assigned to selected nodes.

            • max number Required

              Maximum number of shards in an index, counting only shards assigned to selected nodes.

            • min number Required

              Minimum number of shards in an index, counting only shards assigned to selected nodes.

        • primaries number

          Number of primary shards assigned to selected nodes.

        • replication number

          Ratio of replica shards to primary shards across all selected nodes.

        • total number

          Total number of shards assigned to selected nodes.

      • store object Required
        Hide store attributes Show store attributes object
        • size number | string

        • size_in_bytes number Required

          Total size, in bytes, of all shards assigned to selected nodes.

        • reserved number | string

        • reserved_in_bytes number Required

          A prediction, in bytes, of how much larger the shard stores will eventually grow due to ongoing peer recoveries, restoring snapshots, and similar activities.

        • total_data_set_size number | string

        • total_data_set_size_in_bytes number

          Total data set size, in bytes, of all shards assigned to selected nodes. This includes the size of shards not stored fully on the nodes, such as the cache for partially mounted indices.

      • mappings object Required
        Hide mappings attributes Show mappings attributes object
        • field_types array[object] Required

          Contains statistics about field data types used in selected nodes.

          Hide field_types attributes Show field_types attributes object
          • name string Required
          • count number Required

            The number of occurrences of the field type in selected nodes.

          • index_count number Required

            The number of indices containing the field type in selected nodes.

          • indexed_vector_count number

            For dense_vector field types, number of indexed vector types in selected nodes.

          • indexed_vector_dim_max number

            For dense_vector field types, the maximum dimension of all indexed vector types in selected nodes.

          • indexed_vector_dim_min number

            For dense_vector field types, the minimum dimension of all indexed vector types in selected nodes.

          • script_count number Generally available; Added in 7.13.0

            The number of fields that declare a script.

        • runtime_field_types array[object]

          Contains statistics about runtime field data types used in selected nodes.

          Hide runtime_field_types attributes Show runtime_field_types attributes object
          • chars_max number Required

            Maximum number of characters for a single runtime field script.

          • chars_total number Required

            Total number of characters for the scripts that define the current runtime field data type.

          • count number Required

            Number of runtime fields mapped to the field data type in selected nodes.

          • doc_max number Required

            Maximum number of accesses to doc_values for a single runtime field script

          • doc_total number Required

            Total number of accesses to doc_values for the scripts that define the current runtime field data type.

          • index_count number Required

            Number of indices containing a mapping of the runtime field data type in selected nodes.

          • lang array[string] Required

            Script languages used for the runtime fields scripts.

          • lines_max number Required

            Maximum number of lines for a single runtime field script.

          • lines_total number Required

            Total number of lines for the scripts that define the current runtime field data type.

          • name string Required
          • scriptless_count number Required

            Number of runtime fields that don’t declare a script.

          • shadowed_count number Required

            Number of runtime fields that shadow an indexed field.

          • source_max number Required

            Maximum number of accesses to _source for a single runtime field script.

          • source_total number Required

            Total number of accesses to _source for the scripts that define the current runtime field data type.

        • total_field_count number

          Total number of fields in all non-system indices.

        • total_deduplicated_field_count number

          Total number of fields in all non-system indices, accounting for mapping deduplication.

        • total_deduplicated_mapping_size number | string

        • total_deduplicated_mapping_size_in_bytes number

          Total size of all mappings, in bytes, after deduplication and compression.

      • versions array[object]

        Contains statistics about analyzers and analyzer components used in selected nodes.

        Hide versions attributes Show versions attributes object
        • index_count number Required
        • primary_shard_count number Required
        • total_primary_bytes number Required
        • version string Required
    • nodes object Required
      Hide nodes attributes Show nodes attributes object
      • count object Required
        Hide count attributes Show count attributes object
        • coordinating_only number Required
        • data number Required
        • data_cold number Required
        • data_content number Required
        • data_frozen number Generally available; Added in 7.13.0
        • data_hot number Required
        • data_warm number Required
        • ingest number Required
        • master number Required
        • ml number Required
        • remote_cluster_client number Required
        • total number Required
        • transform number Required
        • voting_only number Required
      • discovery_types object Required

        Contains statistics about the discovery types used by selected nodes.

        Hide discovery_types attribute Show discovery_types attribute object
        • * number Additional properties
      • fs object Required
        Hide fs attributes Show fs attributes object
        • available_in_bytes number Required

          Total number of bytes available to JVM in file stores across all selected nodes. Depending on operating system or process-level restrictions, this number may be less than nodes.fs.free_in_byes. This is the actual amount of free disk space the selected Elasticsearch nodes can use.

        • free_in_bytes number Required

          Total number of unallocated bytes in file stores across all selected nodes.

        • total_in_bytes number Required

          Total size, in bytes, of all file stores across all selected nodes.

      • indexing_pressure object Required
        Hide indexing_pressure attribute Show indexing_pressure attribute object
        • memory object Required
          Hide memory attributes Show memory attributes object
          • current object Required
            Hide current attributes Show current attributes object
            • all_in_bytes number Required
            • combined_coordinating_and_primary_in_bytes number Required
            • coordinating_in_bytes number Required
            • coordinating_rejections number
            • primary_in_bytes number Required
            • primary_rejections number
            • replica_in_bytes number Required
            • replica_rejections number
          • limit_in_bytes number Required
          • total object Required
            Hide total attributes Show total attributes object
            • all_in_bytes number Required
            • combined_coordinating_and_primary_in_bytes number Required
            • coordinating_in_bytes number Required
            • coordinating_rejections number
            • primary_in_bytes number Required
            • primary_rejections number
            • replica_in_bytes number Required
            • replica_rejections number
      • ingest object Required
        Hide ingest attributes Show ingest attributes object
        • number_of_pipelines number Required
        • processor_stats object Required
          Hide processor_stats attribute Show processor_stats attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • count number Required
            • current number Required
            • failed number Required
            • time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • jvm object Required
        Hide jvm attributes Show jvm attributes object
        • Time unit for milliseconds

        • mem object Required
          Hide mem attributes Show mem attributes object
          • heap_max_in_bytes number Required

            Maximum amount of memory, in bytes, available for use by the heap across all selected nodes.

          • heap_used_in_bytes number Required

            Memory, in bytes, currently in use by the heap across all selected nodes.

        • threads number Required

          Number of active threads in use by JVM across all selected nodes.

        • versions array[object] Required

          Contains statistics about the JVM versions used by selected nodes.

          Hide versions attributes Show versions attributes object
          • bundled_jdk boolean Required

            Always true. All distributions come with a bundled Java Development Kit (JDK).

          • count number Required

            Total number of selected nodes using JVM.

          • using_bundled_jdk boolean Required

            If true, a bundled JDK is in use by JVM.

          • version string Required
          • vm_name string Required

            Name of the JVM.

          • vm_vendor string Required

            Vendor of the JVM.

          • vm_version string Required
      • network_types object Required
        Hide network_types attributes Show network_types attributes object
        • http_types object Required

          Contains statistics about the HTTP network types used by selected nodes.

          Hide http_types attribute Show http_types attribute object
          • * number Additional properties
        • transport_types object Required

          Contains statistics about the transport network types used by selected nodes.

          Hide transport_types attribute Show transport_types attribute object
          • * number Additional properties
      • os object Required
        Hide os attributes Show os attributes object
        • allocated_processors number Required

          Number of processors used to calculate thread pool size across all selected nodes. This number can be set with the processors setting of a node and defaults to the number of processors reported by the operating system. In both cases, this number will never be larger than 32.

        • architectures array[object]

          Contains statistics about processor architectures (for example, x86_64 or aarch64) used by selected nodes.

          Hide architectures attributes Show architectures attributes object
          • arch string Required

            Name of an architecture used by one or more selected nodes.

          • count number Required

            Number of selected nodes using the architecture.

        • available_processors number Required

          Number of processors available to JVM across all selected nodes.

        • mem object Required
          Hide mem attributes Show mem attributes object
          • adjusted_total_in_bytes number Generally available; Added in 7.16.0

            Total amount, in bytes, of memory across all selected nodes, but using the value specified using the es.total_memory_bytes system property instead of measured total memory for those nodes where that system property was set.

          • free_in_bytes number Required

            Amount, in bytes, of free physical memory across all selected nodes.

          • free_percent number Required

            Percentage of free physical memory across all selected nodes.

          • total_in_bytes number Required

            Total amount, in bytes, of physical memory across all selected nodes.

          • used_in_bytes number Required

            Amount, in bytes, of physical memory in use across all selected nodes.

          • used_percent number Required

            Percentage of physical memory in use across all selected nodes.

        • names array[object] Required

          Contains statistics about operating systems used by selected nodes.

          Hide names attributes Show names attributes object
          • count number Required

            Number of selected nodes using the operating system.

          • name string Required
        • pretty_names array[object] Required

          Contains statistics about operating systems used by selected nodes.

          Hide pretty_names attributes Show pretty_names attributes object
          • count number Required

            Number of selected nodes using the operating system.

          • pretty_name string Required
      • packaging_types array[object] Required

        Contains statistics about Elasticsearch distributions installed on selected nodes.

        Hide packaging_types attributes Show packaging_types attributes object
        • count number Required

          Number of selected nodes using the distribution flavor and file type.

        • flavor string Required

          Type of Elasticsearch distribution. This is always default.

        • type string Required

          File type (such as tar or zip) used for the distribution package.

      • plugins array[object] Required

        Contains statistics about installed plugins and modules by selected nodes. If no plugins or modules are installed, this array is empty.

        Hide plugins attributes Show plugins attributes object
        • classname string Required
        • description string Required
        • elasticsearch_version string Required
        • extended_plugins array[string] Required
        • has_native_controller boolean Required
        • java_version string Required
        • name string Required
        • version string Required
        • licensed boolean Required
      • process object Required
        Hide process attributes Show process attributes object
        • cpu object Required
          Hide cpu attribute Show cpu attribute object
          • percent number Required

            Percentage of CPU used across all selected nodes. Returns -1 if not supported.

        • open_file_descriptors object Required
          Hide open_file_descriptors attributes Show open_file_descriptors attributes object
          • avg number Required

            Average number of concurrently open file descriptors. Returns -1 if not supported.

          • max number Required

            Maximum number of concurrently open file descriptors allowed across all selected nodes. Returns -1 if not supported.

          • min number Required

            Minimum number of concurrently open file descriptors across all selected nodes. Returns -1 if not supported.

      • versions array[string] Required

        Array of Elasticsearch versions used on selected nodes.

    • status string Required

      Values are green, GREEN, yellow, YELLOW, red, or RED.

    • timestamp number Required

      Unix timestamp, in milliseconds, for the last time the cluster statistics were refreshed.

    • ccs object Required
      Hide ccs attributes Show ccs attributes object
      • clusters object

        Contains remote cluster settings and metrics collected from them. The keys are cluster names, and the values are per-cluster data. Only present if include_remotes option is set to true.

        Hide clusters attribute Show clusters attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • cluster_uuid string Required

            The UUID of the remote cluster.

          • mode string Required

            The connection mode used to communicate with the remote cluster.

          • skip_unavailable boolean Required

            The skip_unavailable setting used for this remote cluster.

          • transport_compress string Required

            Transport compression setting used for this remote cluster.

          • status string Required

            Values are green, GREEN, yellow, YELLOW, red, or RED.

          • version array[string] Required

            The list of Elasticsearch versions used by the nodes on the remote cluster.

          • nodes_count number Required

            The total count of nodes in the remote cluster.

          • shards_count number Required

            The total number of shards in the remote cluster.

          • indices_count number Required

            The total number of indices in the remote cluster.

          • indices_total_size_in_bytes number Required

            Total data set size, in bytes, of all shards assigned to selected nodes.

          • indices_total_size string

            Total data set size of all shards assigned to selected nodes, as a human-readable string.

          • max_heap_in_bytes number Required

            Maximum amount of memory, in bytes, available for use by the heap across the nodes of the remote cluster.

          • max_heap string

            Maximum amount of memory available for use by the heap across the nodes of the remote cluster, as a human-readable string.

          • mem_total_in_bytes number Required

            Total amount, in bytes, of physical memory across the nodes of the remote cluster.

          • mem_total string

            Total amount of physical memory across the nodes of the remote cluster, as a human-readable string.

      • _esql object
        Hide _esql attributes Show _esql attributes object
        • total number Required

          The total number of cross-cluster search requests that have been executed by the cluster.

        • success number Required

          The total number of cross-cluster search requests that have been successfully executed by the cluster.

        • skipped number Required

          The total number of cross-cluster search requests (successful or failed) that had at least one remote cluster skipped.

        • took object Required
          Hide took attributes Show took attributes object
          • Time unit for milliseconds

          • Time unit for milliseconds

          • Time unit for milliseconds

        • took_mrt_true object
          Hide took_mrt_true attributes Show took_mrt_true attributes object
          • Time unit for milliseconds

          • Time unit for milliseconds

          • Time unit for milliseconds

        • took_mrt_false object
          Hide took_mrt_false attributes Show took_mrt_false attributes object
          • Time unit for milliseconds

          • Time unit for milliseconds

          • Time unit for milliseconds

        • remotes_per_search_max number Required

          The maximum number of remote clusters that were queried in a single cross-cluster search request.

        • remotes_per_search_avg number Required

          The average number of remote clusters that were queried in a single cross-cluster search request.

        • failure_reasons object Required

          Statistics about the reasons for cross-cluster search request failures. The keys are the failure reason names and the values are the number of requests that failed for that reason.

          Hide failure_reasons attribute Show failure_reasons attribute object
          • * number Additional properties
        • features object Required

          The keys are the names of the search feature, and the values are the number of requests that used that feature. Single request can use more than one feature (e.g. both async and wildcard).

          Hide features attribute Show features attribute object
          • * number Additional properties
        • clients object Required

          Statistics about the clients that executed cross-cluster search requests. The keys are the names of the clients, and the values are the number of requests that were executed by that client. Only known clients (such as kibana or elasticsearch) are counted.

          Hide clients attribute Show clients attribute object
          • * number Additional properties
        • clusters object Required

          Statistics about the clusters that were queried in cross-cluster search requests. The keys are cluster names, and the values are per-cluster telemetry data. This also includes the local cluster itself, which uses the name (local).

          Hide clusters attribute Show clusters attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • total number Required

              The total number of successful (not skipped) cross-cluster search requests that were executed against this cluster. This may include requests where partial results were returned, but not requests in which the cluster has been skipped entirely.

            • skipped number Required

              The total number of cross-cluster search requests for which this cluster was skipped.

            • took object Required
GET /_cluster/stats/nodes/{node_id}
curl \
 --request GET 'http://api.example.com/_cluster/stats/nodes/{node_id}' \
 --header "Authorization: $API_KEY"




Clear the archived repositories metering Technical preview; Added in 7.16.0

DELETE /_nodes/{node_id}/_repositories_metering/{max_archive_version}

Clear the archived repositories metering information in the cluster.

Required authorization

  • Cluster privileges: monitor,manage

Path parameters

  • node_id string | array[string] Required

    Comma-separated list of node IDs or names used to limit returned information.

  • max_archive_version number Required

    Specifies the maximum archive_version to be cleared from the archive.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object

      Contains statistics about the number of nodes selected by the request.

      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide failures attributes Show failures attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • nodes object Required

      Contains repositories metering information for the nodes selected by the request.

      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • repository_name string Required
        • repository_type string Required

          Repository type.

        • repository_location object Required
          Hide repository_location attributes Show repository_location attributes object
          • base_path string Required
          • container string

            Container name (Azure)

          • bucket string

            Bucket name (GCP, S3)

        • repository_ephemeral_id string Required
        • Time unit for milliseconds

        • Time unit for milliseconds

        • archived boolean Required

          A flag that tells whether or not this object has been archived. When a repository is closed or updated the repository metering information is archived and kept for a certain period of time. This allows retrieving the repository metering information of previous repository instantiations.

        • cluster_version number
        • request_counts object Required
          Hide request_counts attributes Show request_counts attributes object
          • GetBlobProperties number

            Number of Get Blob Properties requests (Azure)

          • GetBlob number

            Number of Get Blob requests (Azure)

          • ListBlobs number

            Number of List Blobs requests (Azure)

          • PutBlob number

            Number of Put Blob requests (Azure)

          • PutBlock number

            Number of Put Block (Azure)

          • PutBlockList number

            Number of Put Block List requests

          • GetObject number

            Number of get object requests (GCP, S3)

          • ListObjects number

            Number of list objects requests (GCP, S3)

          • InsertObject number

            Number of insert object requests, including simple, multipart and resumable uploads. Resumable uploads can perform multiple http requests to insert a single object but they are considered as a single request since they are billed as an individual operation. (GCP)

          • PutObject number

            Number of PutObject requests (S3)

          • PutMultipartObject number

            Number of Multipart requests, including CreateMultipartUpload, UploadPart and CompleteMultipartUpload requests (S3)

DELETE /_nodes/{node_id}/_repositories_metering/{max_archive_version}
curl \
 --request DELETE 'http://api.example.com/_nodes/{node_id}/_repositories_metering/{max_archive_version}' \
 --header "Authorization: $API_KEY"

Get cluster repositories metering Technical preview; Added in 7.16.0

GET /_nodes/{node_id}/_repositories_metering

Get repositories metering information for a cluster. This API exposes monotonically non-decreasing counters and it is expected that clients would durably store the information needed to compute aggregations over a period of time. Additionally, the information exposed by this API is volatile, meaning that it will not be present after node restarts.

Required authorization

  • Cluster privileges: monitor,manage

Path parameters

  • node_id string | array[string] Required

    Comma-separated list of node IDs or names used to limit returned information. For more information about the nodes selective options, refer to the node specification documentation.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object

      Contains statistics about the number of nodes selected by the request.

      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide failures attributes Show failures attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • nodes object Required

      Contains repositories metering information for the nodes selected by the request.

      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • repository_name string Required
        • repository_type string Required

          Repository type.

        • repository_location object Required
          Hide repository_location attributes Show repository_location attributes object
          • base_path string Required
          • container string

            Container name (Azure)

          • bucket string

            Bucket name (GCP, S3)

        • repository_ephemeral_id string Required
        • Time unit for milliseconds

        • Time unit for milliseconds

        • archived boolean Required

          A flag that tells whether or not this object has been archived. When a repository is closed or updated the repository metering information is archived and kept for a certain period of time. This allows retrieving the repository metering information of previous repository instantiations.

        • cluster_version number
        • request_counts object Required
          Hide request_counts attributes Show request_counts attributes object
          • GetBlobProperties number

            Number of Get Blob Properties requests (Azure)

          • GetBlob number

            Number of Get Blob requests (Azure)

          • ListBlobs number

            Number of List Blobs requests (Azure)

          • PutBlob number

            Number of Put Blob requests (Azure)

          • PutBlock number

            Number of Put Block (Azure)

          • PutBlockList number

            Number of Put Block List requests

          • GetObject number

            Number of get object requests (GCP, S3)

          • ListObjects number

            Number of list objects requests (GCP, S3)

          • InsertObject number

            Number of insert object requests, including simple, multipart and resumable uploads. Resumable uploads can perform multiple http requests to insert a single object but they are considered as a single request since they are billed as an individual operation. (GCP)

          • PutObject number

            Number of PutObject requests (S3)

          • PutMultipartObject number

            Number of Multipart requests, including CreateMultipartUpload, UploadPart and CompleteMultipartUpload requests (S3)

GET /_nodes/{node_id}/_repositories_metering
curl \
 --request GET 'http://api.example.com/_nodes/{node_id}/_repositories_metering' \
 --header "Authorization: $API_KEY"

Get the hot threads for nodes Generally available

GET /_nodes/hot_threads

Get a breakdown of the hot threads on each selected node in the cluster. The output is plain text with a breakdown of the top hot threads for each node.

Required authorization

  • Cluster privileges: monitor,manage

Query parameters

  • ignore_idle_threads boolean

    If true, known idle threads (e.g. waiting in a socket select, or to get a task from an empty queue) are filtered out.

  • interval string

    The interval to do the second sampling of threads.

    Values are -1 or 0.

  • snapshots number

    Number of samples of thread stacktrace.

  • threads number

    Specifies the number of hot threads to provide information for.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • type string

    The type to sample.

    Values are cpu, wait, block, gpu, or mem.

  • sort string

    The sort order for 'cpu' type (default: total)

    Values are cpu, wait, block, gpu, or mem.

Responses

  • 200 application/json
GET /_nodes/hot_threads
curl \
 --request GET 'http://api.example.com/_nodes/hot_threads' \
 --header "Authorization: $API_KEY"








Get node information Generally available; Added in 1.3.0

GET /_nodes/{node_id}

By default, the API returns all attributes and core settings for cluster nodes.

Path parameters

  • node_id string | array[string] Required

    Comma-separated list of node IDs or names used to limit returned information.

Query parameters

  • flat_settings boolean

    If true, returns settings in flat format.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object

      Contains statistics about the number of nodes selected by the request.

      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide failures attributes Show failures attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • nodes object Required
      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • attributes object Required
          Hide attributes attribute Show attributes attribute object
          • * string Additional properties
        • build_flavor string Required
        • build_hash string Required

          Short hash of the last git commit in this release.

        • build_type string Required
        • host string Required
        • http object
          Hide http attributes Show http attributes object
        • ip string Required
        • jvm object
          Hide jvm attributes Show jvm attributes object
          • gc_collectors array[string] Required
          • mem object Required
            Hide mem attributes Show mem attributes object
            • direct_max
            • direct_max_in_bytes number Required
            • heap_init
            • heap_init_in_bytes number Required
            • heap_max
            • heap_max_in_bytes number Required
            • non_heap_init
            • non_heap_init_in_bytes number Required
            • non_heap_max
            • non_heap_max_in_bytes number Required
          • memory_pools array[string] Required
          • pid number Required
          • Time unit for milliseconds

          • version string Required
          • vm_name string Required
          • vm_vendor string Required
          • vm_version string Required
          • using_bundled_jdk boolean Required
          • using_compressed_ordinary_object_pointers boolean | string

          • input_arguments array[string] Required
        • name string Required
        • network object
          Hide network attributes Show network attributes object
          • primary_interface object Required
            Hide primary_interface attributes Show primary_interface attributes object
            • address string Required
            • mac_address string Required
            • name string Required
          • refresh_interval number Required
        • os object
          Hide os attributes Show os attributes object
          • arch string Required

            Name of the JVM architecture (ex: amd64, x86)

          • available_processors number Required

            Number of processors available to the Java virtual machine

          • allocated_processors number

            The number of processors actually used to calculate thread pool size. This number can be set with the node.processors setting of a node and defaults to the number of processors reported by the OS.

          • name string Required
          • pretty_name string Required
          • Time unit for milliseconds

          • version string Required
          • cpu object
            Hide cpu attributes Show cpu attributes object
            • cache_size string Required
            • cache_size_in_bytes number Required
            • cores_per_socket number Required
            • mhz number Required
            • model string Required
            • total_cores number Required
            • total_sockets number Required
            • vendor string Required
          • mem object
            Hide mem attributes Show mem attributes object
            • total string Required
            • total_in_bytes number Required
          • swap object
            Hide swap attributes Show swap attributes object
            • total string Required
            • total_in_bytes number Required
        • plugins array[object]
          Hide plugins attributes Show plugins attributes object
          • classname string Required
          • description string Required
          • elasticsearch_version string Required
          • extended_plugins array[string] Required
          • has_native_controller boolean Required
          • java_version string Required
          • name string Required
          • version string Required
          • licensed boolean Required
        • process object
          Hide process attributes Show process attributes object
          • id number Required

            Process identifier (PID)

          • mlockall boolean Required

            Indicates if the process address space has been successfully locked in memory

          • Time unit for milliseconds

        • roles array[string] Required

          Values are master, data, data_cold, data_content, data_frozen, data_hot, data_warm, client, ingest, ml, voting_only, transform, remote_cluster_client, or coordinating_only.

        • settings object
          Hide settings attributes Show settings attributes object
          • cluster object Required
            Hide cluster attributes Show cluster attributes object
            • name string Required
            • routing object
            • election object Required
            • initial_master_nodes array[string]
            • deprecation_indexing object
          • node object Required
            Hide node attributes Show node attributes object
            • name string Required
            • attr object Required
            • max_local_storage_nodes string
          • path object
            Hide path attributes Show path attributes object
            • logs string
            • home string
            • repo array[string]
            • data
          • repositories object
            Hide repositories attribute Show repositories attribute object
            • url object Required
          • discovery object
            Hide discovery attributes Show discovery attributes object
            • seed_hosts array[string]
            • type string
            • seed_providers array[string]
          • action object
            Hide action attribute Show action attribute object
            • destructive_requires_name string Required
          • client object
            Hide client attribute Show client attribute object
            • type string Required
          • http object Required
            Hide http attributes Show http attributes object
            • type object Required
            • type.default string
            • compression
            • port
          • bootstrap object
            Hide bootstrap attribute Show bootstrap attribute object
            • memory_lock string Required
          • transport object Required
            Hide transport attributes Show transport attributes object
            • type object Required
            • type.default string
            • features object
          • network object
            Hide network attribute Show network attribute object
            • host
          • xpack object
            Hide xpack attributes Show xpack attributes object
            • license object
            • security object Required
            • notification object
            • ml object
          • script object
            Hide script attributes Show script attributes object
            • allowed_types string Required
            • disable_max_compilations_rate string
          • ingest object
            Hide ingest attributes Show ingest attributes object
            • attachment object
            • append object
            • csv object
            • convert object
            • date object
            • date_index_name object
            • dot_expander object
            • enrich object
            • fail object
            • foreach object
            • json object
            • user_agent object
            • kv object
            • geoip object
            • grok object
            • gsub object
            • join object
            • lowercase object
            • remove object
            • rename object
            • script object
            • set object
            • sort object
            • split object
            • trim object
            • uppercase object
            • urldecode object
            • bytes object
            • dissect object
            • set_security_user object
            • pipeline object
            • drop object
            • circle object
            • inference object
        • thread_pool object
          Hide thread_pool attribute Show thread_pool attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • core number
            • keep_alive string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • max number
            • queue_size number Required
            • size number
            • type string Required
        • total_indexing_buffer number

          Total heap allowed to be used to hold recently indexed documents before they must be written to disk. This size is a shared pool across all shards on this node, and is controlled by Indexing Buffer settings.

        • total_indexing_buffer_in_bytes number | string

        • transport object
          Hide transport attributes Show transport attributes object
          • bound_address array[string] Required
          • publish_address string Required
          • profiles object Required
            Hide profiles attribute Show profiles attribute object
            • * string Additional properties
        • transport_address string Required
        • version string Required
        • modules array[object]
          Hide modules attributes Show modules attributes object
          • classname string Required
          • description string Required
          • elasticsearch_version string Required
          • extended_plugins array[string] Required
          • has_native_controller boolean Required
          • java_version string Required
          • name string Required
          • version string Required
          • licensed boolean Required
        • ingest object
          Hide ingest attribute Show ingest attribute object
          • processors array[object] Required
        • aggregations object
          Hide aggregations attribute Show aggregations attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • types array[string] Required
GET /_nodes/{node_id}
curl \
 --request GET 'http://api.example.com/_nodes/{node_id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
An abbreviated response when requesting cluster nodes information.
{
    "_nodes": {},
    "cluster_name": "elasticsearch",
    "nodes": {
      "USpTGYaBSIKbgSUJR2Z9lg": {
        "name": "node-0",
        "transport_address": "192.168.17:9300",
        "host": "node-0.elastic.co",
        "ip": "192.168.17",
        "version": "{version}",
        "transport_version": 100000298,
        "index_version": 100000074,
        "component_versions": {
          "ml_config_version": 100000162,
          "transform_config_version": 100000096
        },
        "build_flavor": "default",
        "build_type": "{build_type}",
        "build_hash": "587409e",
        "roles": [
          "master",
          "data",
          "ingest"
        ],
        "attributes": {},
        "plugins": [
          {
            "name": "analysis-icu",
            "version": "{version}",
            "description": "The ICU Analysis plugin integrates Lucene ICU
  module into elasticsearch, adding ICU relates analysis components.",
            "classname":
  "org.elasticsearch.plugin.analysis.icu.AnalysisICUPlugin",
            "has_native_controller": false
          }
        ],
        "modules": [
          {
            "name": "lang-painless",
            "version": "{version}",
            "description": "An easy, safe and fast scripting language for
  Elasticsearch",
            "classname": "org.elasticsearch.painless.PainlessPlugin",
            "has_native_controller": false
          }
        ]
      }
    }
}




Get node information Generally available; Added in 1.3.0

GET /_nodes/{node_id}/{metric}

By default, the API returns all attributes and core settings for cluster nodes.

Path parameters

  • node_id string | array[string] Required

    Comma-separated list of node IDs or names used to limit returned information.

  • metric string | array[string] Required

    Limits the information returned to the specific metrics. Supports a comma-separated list, such as http,ingest.

Query parameters

  • flat_settings boolean

    If true, returns settings in flat format.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object

      Contains statistics about the number of nodes selected by the request.

      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide failures attributes Show failures attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • nodes object Required
      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • attributes object Required
          Hide attributes attribute Show attributes attribute object
          • * string Additional properties
        • build_flavor string Required
        • build_hash string Required

          Short hash of the last git commit in this release.

        • build_type string Required
        • host string Required
        • http object
          Hide http attributes Show http attributes object
        • ip string Required
        • jvm object
          Hide jvm attributes Show jvm attributes object
          • gc_collectors array[string] Required
          • mem object Required
            Hide mem attributes Show mem attributes object
            • direct_max
            • direct_max_in_bytes number Required
            • heap_init
            • heap_init_in_bytes number Required
            • heap_max
            • heap_max_in_bytes number Required
            • non_heap_init
            • non_heap_init_in_bytes number Required
            • non_heap_max
            • non_heap_max_in_bytes number Required
          • memory_pools array[string] Required
          • pid number Required
          • Time unit for milliseconds

          • version string Required
          • vm_name string Required
          • vm_vendor string Required
          • vm_version string Required
          • using_bundled_jdk boolean Required
          • using_compressed_ordinary_object_pointers boolean | string

          • input_arguments array[string] Required
        • name string Required
        • network object
          Hide network attributes Show network attributes object
          • primary_interface object Required
            Hide primary_interface attributes Show primary_interface attributes object
            • address string Required
            • mac_address string Required
            • name string Required
          • refresh_interval number Required
        • os object
          Hide os attributes Show os attributes object
          • arch string Required

            Name of the JVM architecture (ex: amd64, x86)

          • available_processors number Required

            Number of processors available to the Java virtual machine

          • allocated_processors number

            The number of processors actually used to calculate thread pool size. This number can be set with the node.processors setting of a node and defaults to the number of processors reported by the OS.

          • name string Required
          • pretty_name string Required
          • Time unit for milliseconds

          • version string Required
          • cpu object
            Hide cpu attributes Show cpu attributes object
            • cache_size string Required
            • cache_size_in_bytes number Required
            • cores_per_socket number Required
            • mhz number Required
            • model string Required
            • total_cores number Required
            • total_sockets number Required
            • vendor string Required
          • mem object
            Hide mem attributes Show mem attributes object
            • total string Required
            • total_in_bytes number Required
          • swap object
            Hide swap attributes Show swap attributes object
            • total string Required
            • total_in_bytes number Required
        • plugins array[object]
          Hide plugins attributes Show plugins attributes object
          • classname string Required
          • description string Required
          • elasticsearch_version string Required
          • extended_plugins array[string] Required
          • has_native_controller boolean Required
          • java_version string Required
          • name string Required
          • version string Required
          • licensed boolean Required
        • process object
          Hide process attributes Show process attributes object
          • id number Required

            Process identifier (PID)

          • mlockall boolean Required

            Indicates if the process address space has been successfully locked in memory

          • Time unit for milliseconds

        • roles array[string] Required

          Values are master, data, data_cold, data_content, data_frozen, data_hot, data_warm, client, ingest, ml, voting_only, transform, remote_cluster_client, or coordinating_only.

        • settings object
          Hide settings attributes Show settings attributes object
          • cluster object Required
            Hide cluster attributes Show cluster attributes object
            • name string Required
            • routing object
            • election object Required
            • initial_master_nodes array[string]
            • deprecation_indexing object
          • node object Required
            Hide node attributes Show node attributes object
            • name string Required
            • attr object Required
            • max_local_storage_nodes string
          • path object
            Hide path attributes Show path attributes object
            • logs string
            • home string
            • repo array[string]
            • data
          • repositories object
            Hide repositories attribute Show repositories attribute object
            • url object Required
          • discovery object
            Hide discovery attributes Show discovery attributes object
            • seed_hosts array[string]
            • type string
            • seed_providers array[string]
          • action object
            Hide action attribute Show action attribute object
            • destructive_requires_name string Required
          • client object
            Hide client attribute Show client attribute object
            • type string Required
          • http object Required
            Hide http attributes Show http attributes object
            • type object Required
            • type.default string
            • compression
            • port
          • bootstrap object
            Hide bootstrap attribute Show bootstrap attribute object
            • memory_lock string Required
          • transport object Required
            Hide transport attributes Show transport attributes object
            • type object Required
            • type.default string
            • features object
          • network object
            Hide network attribute Show network attribute object
            • host
          • xpack object
            Hide xpack attributes Show xpack attributes object
            • license object
            • security object Required
            • notification object
            • ml object
          • script object
            Hide script attributes Show script attributes object
            • allowed_types string Required
            • disable_max_compilations_rate string
          • ingest object
            Hide ingest attributes Show ingest attributes object
            • attachment object
            • append object
            • csv object
            • convert object
            • date object
            • date_index_name object
            • dot_expander object
            • enrich object
            • fail object
            • foreach object
            • json object
            • user_agent object
            • kv object
            • geoip object
            • grok object
            • gsub object
            • join object
            • lowercase object
            • remove object
            • rename object
            • script object
            • set object
            • sort object
            • split object
            • trim object
            • uppercase object
            • urldecode object
            • bytes object
            • dissect object
            • set_security_user object
            • pipeline object
            • drop object
            • circle object
            • inference object
        • thread_pool object
          Hide thread_pool attribute Show thread_pool attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • core number
            • keep_alive string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • max number
            • queue_size number Required
            • size number
            • type string Required
        • total_indexing_buffer number

          Total heap allowed to be used to hold recently indexed documents before they must be written to disk. This size is a shared pool across all shards on this node, and is controlled by Indexing Buffer settings.

        • total_indexing_buffer_in_bytes number | string

        • transport object
          Hide transport attributes Show transport attributes object
          • bound_address array[string] Required
          • publish_address string Required
          • profiles object Required
            Hide profiles attribute Show profiles attribute object
            • * string Additional properties
        • transport_address string Required
        • version string Required
        • modules array[object]
          Hide modules attributes Show modules attributes object
          • classname string Required
          • description string Required
          • elasticsearch_version string Required
          • extended_plugins array[string] Required
          • has_native_controller boolean Required
          • java_version string Required
          • name string Required
          • version string Required
          • licensed boolean Required
        • ingest object
          Hide ingest attribute Show ingest attribute object
          • processors array[object] Required
        • aggregations object
          Hide aggregations attribute Show aggregations attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • types array[string] Required
GET /_nodes/{node_id}/{metric}
curl \
 --request GET 'http://api.example.com/_nodes/{node_id}/{metric}' \
 --header "Authorization: $API_KEY"
Response examples (200)
An abbreviated response when requesting cluster nodes information.
{
    "_nodes": {},
    "cluster_name": "elasticsearch",
    "nodes": {
      "USpTGYaBSIKbgSUJR2Z9lg": {
        "name": "node-0",
        "transport_address": "192.168.17:9300",
        "host": "node-0.elastic.co",
        "ip": "192.168.17",
        "version": "{version}",
        "transport_version": 100000298,
        "index_version": 100000074,
        "component_versions": {
          "ml_config_version": 100000162,
          "transform_config_version": 100000096
        },
        "build_flavor": "default",
        "build_type": "{build_type}",
        "build_hash": "587409e",
        "roles": [
          "master",
          "data",
          "ingest"
        ],
        "attributes": {},
        "plugins": [
          {
            "name": "analysis-icu",
            "version": "{version}",
            "description": "The ICU Analysis plugin integrates Lucene ICU
  module into elasticsearch, adding ICU relates analysis components.",
            "classname":
  "org.elasticsearch.plugin.analysis.icu.AnalysisICUPlugin",
            "has_native_controller": false
          }
        ],
        "modules": [
          {
            "name": "lang-painless",
            "version": "{version}",
            "description": "An easy, safe and fast scripting language for
  Elasticsearch",
            "classname": "org.elasticsearch.painless.PainlessPlugin",
            "has_native_controller": false
          }
        ]
      }
    }
}

Reload the keystore on nodes in the cluster Generally available; Added in 6.5.0

POST /_nodes/reload_secure_settings

Secure settings are stored in an on-disk keystore. Certain of these settings are reloadable. That is, you can change them on disk and reload them without restarting any nodes in the cluster. When you have updated reloadable secure settings in your keystore, you can use this API to reload those settings on each node.

When the Elasticsearch keystore is password protected and not simply obfuscated, you must provide the password for the keystore when you reload the secure settings. Reloading the settings for the whole cluster assumes that the keystores for all nodes are protected with the same password; this method is allowed only when inter-node communications are encrypted. Alternatively, you can reload the secure settings on each node by locally accessing the API and passing the node-specific Elasticsearch keystore password.

Query parameters

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

application/json

Body

  • secure_settings_password string

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object

      Contains statistics about the number of nodes selected by the request.

      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide failures attributes Show failures attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • nodes object Required
      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • name string Required
        • reload_exception object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reload_exception attributes Show reload_exception attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

POST /_nodes/reload_secure_settings
curl \
 --request POST 'http://api.example.com/_nodes/reload_secure_settings' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"secure_settings_password\": \"keystore-password\"\n}"'
Request example
Run `POST _nodes/reload_secure_settings` to reload the keystore on nodes in the cluster.
{
  "secure_settings_password": "keystore-password"
}
Response examples (200)
A successful response when reloading keystore on nodes in your cluster.
{
  "_nodes": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "cluster_name": "my_cluster",
  "nodes": {
    "pQHNt5rXTTWNvUgOrdynKg": {
      "name": "node-0"
    }
  }
}

Reload the keystore on nodes in the cluster Generally available; Added in 6.5.0

POST /_nodes/{node_id}/reload_secure_settings

Secure settings are stored in an on-disk keystore. Certain of these settings are reloadable. That is, you can change them on disk and reload them without restarting any nodes in the cluster. When you have updated reloadable secure settings in your keystore, you can use this API to reload those settings on each node.

When the Elasticsearch keystore is password protected and not simply obfuscated, you must provide the password for the keystore when you reload the secure settings. Reloading the settings for the whole cluster assumes that the keystores for all nodes are protected with the same password; this method is allowed only when inter-node communications are encrypted. Alternatively, you can reload the secure settings on each node by locally accessing the API and passing the node-specific Elasticsearch keystore password.

Path parameters

  • node_id string | array[string] Required

    The names of particular nodes in the cluster to target.

Query parameters

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

application/json

Body

  • secure_settings_password string

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object

      Contains statistics about the number of nodes selected by the request.

      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide failures attributes Show failures attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • nodes object Required
      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • name string Required
        • reload_exception object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reload_exception attributes Show reload_exception attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

POST /_nodes/{node_id}/reload_secure_settings
curl \
 --request POST 'http://api.example.com/_nodes/{node_id}/reload_secure_settings' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"secure_settings_password\": \"keystore-password\"\n}"'
Request example
Run `POST _nodes/reload_secure_settings` to reload the keystore on nodes in the cluster.
{
  "secure_settings_password": "keystore-password"
}
Response examples (200)
A successful response when reloading keystore on nodes in your cluster.
{
  "_nodes": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "cluster_name": "my_cluster",
  "nodes": {
    "pQHNt5rXTTWNvUgOrdynKg": {
      "name": "node-0"
    }
  }
}

Get node statistics Generally available

GET /_nodes/stats

Get statistics for nodes in a cluster. By default, all stats are returned. You can limit the returned information by using metrics.

Required authorization

  • Cluster privileges: monitor,manage

Query parameters

  • completion_fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in fielddata and suggest statistics.

  • fielddata_fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in fielddata statistics.

  • fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in the statistics.

  • groups boolean

    Comma-separated list of search groups to include in the search statistics.

  • include_segment_file_sizes boolean

    If true, the call reports the aggregated disk usage of each one of the Lucene index files (only applies if segment stats are requested).

  • level string

    Indicates whether statistics are aggregated at the cluster, index, or shard level.

    Values are cluster, indices, or shards.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • types array[string]

    A comma-separated list of document types for the indexing index metric.

  • include_unloaded_segments boolean

    If true, the response includes information from segments that are not loaded into memory.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object

      Contains statistics about the number of nodes selected by the request.

      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide failures attributes Show failures attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string
    • nodes object Required
      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • adaptive_selection object

          Statistics about adaptive replica selection.

          Hide adaptive_selection attribute Show adaptive_selection attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • avg_queue_size number

              The exponentially weighted moving average queue size of search requests on the keyed node.

            • avg_response_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • avg_response_time_ns number

              The exponentially weighted moving average response time, in nanoseconds, of search requests on the keyed node.

            • avg_service_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • avg_service_time_ns number

              The exponentially weighted moving average service time, in nanoseconds, of search requests on the keyed node.

            • outgoing_searches number

              The number of outstanding search requests to the keyed node from the node these stats are for.

            • rank string

              The rank of this node; used for shard selection when routing search requests.

        • breakers object

          Statistics about the field data circuit breaker.

          Hide breakers attribute Show breakers attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • estimated_size string

              Estimated memory used for the operation.

            • estimated_size_in_bytes number

              Estimated memory used, in bytes, for the operation.

            • limit_size string

              Memory limit for the circuit breaker.

            • limit_size_in_bytes number

              Memory limit, in bytes, for the circuit breaker.

            • overhead number

              A constant that all estimates for the circuit breaker are multiplied with to calculate a final estimate.

            • tripped number

              Total number of times the circuit breaker has been triggered and prevented an out of memory error.

        • fs object
          Hide fs attributes Show fs attributes object
          • data array[object]

            List of all file stores.

          • timestamp number

            Last time the file stores statistics were refreshed. Recorded in milliseconds since the Unix Epoch.

          • total object
            Hide total attributes Show total attributes object
            • available string

              Total disk space available to this Java virtual machine on all file stores. Depending on OS or process level restrictions, this might appear less than free. This is the actual amount of free disk space the Elasticsearch node can utilise.

            • available_in_bytes number

              Total number of bytes available to this Java virtual machine on all file stores. Depending on OS or process level restrictions, this might appear less than free_in_bytes. This is the actual amount of free disk space the Elasticsearch node can utilise.

            • free string

              Total unallocated disk space in all file stores.

            • free_in_bytes number

              Total number of unallocated bytes in all file stores.

            • total string

              Total size of all file stores.

            • total_in_bytes number

              Total size of all file stores in bytes.

          • io_stats object
            Hide io_stats attributes Show io_stats attributes object
            • devices array[object]

              Array of disk metrics for each device that is backing an Elasticsearch data path. These disk metrics are probed periodically and averages between the last probe and the current probe are computed.

            • total object
        • host string
        • http object
          Hide http attributes Show http attributes object
          • current_open number

            Current number of open HTTP connections for the node.

          • total_opened number

            Total number of HTTP connections opened for the node.

          • clients array[object]

            Information on current and recently-closed HTTP client connections. Clients that have been closed longer than the http.client_stats.closed_channels.max_age setting will not be represented here.

          • routes object Required Generally available; Added in 8.12.0

            Detailed HTTP stats broken down by route

            Hide routes attribute Show routes attribute object
            • * object Additional properties
        • ingest object
          Hide ingest attributes Show ingest attributes object
          • pipelines object

            Contains statistics about ingest pipelines for the node.

            Hide pipelines attribute Show pipelines attribute object
            • * object Additional properties
          • total object
            Hide total attributes Show total attributes object
            • count number Required

              Total number of documents ingested during the lifetime of this node.

            • current number Required

              Total number of documents currently being ingested.

            • failed number Required

              Total number of failed ingest operations during the lifetime of this node.

        • ip string | array[string]

          IP address and port for the node.

        • jvm object
          Hide jvm attributes Show jvm attributes object
          • buffer_pools object

            Contains statistics about JVM buffer pools for the node.

            Hide buffer_pools attribute Show buffer_pools attribute object
            • * object Additional properties
          • classes object
            Hide classes attributes Show classes attributes object
            • current_loaded_count number

              Number of classes currently loaded by JVM.

            • total_loaded_count number

              Total number of classes loaded since the JVM started.

            • total_unloaded_count number

              Total number of classes unloaded since the JVM started.

          • gc object
            Hide gc attribute Show gc attribute object
            • collectors object

              Contains statistics about JVM garbage collectors for the node.

          • mem object
            Hide mem attributes Show mem attributes object
            • heap_used_in_bytes number

              Memory, in bytes, currently in use by the heap.

            • heap_used_percent number

              Percentage of memory currently in use by the heap.

            • heap_committed_in_bytes number

              Amount of memory, in bytes, available for use by the heap.

            • heap_max_in_bytes number

              Maximum amount of memory, in bytes, available for use by the heap.

            • non_heap_used_in_bytes number

              Non-heap memory used, in bytes.

            • non_heap_committed_in_bytes number

              Amount of non-heap memory available, in bytes.

            • pools object

              Contains statistics about heap memory usage for the node.

          • threads object
            Hide threads attributes Show threads attributes object
            • count number

              Number of active threads in use by JVM.

            • peak_count number

              Highest number of threads used by JVM.

          • timestamp number

            Last time JVM statistics were refreshed.

          • uptime string

            Human-readable JVM uptime. Only returned if the human query parameter is true.

          • uptime_in_millis number

            JVM uptime in milliseconds.

        • name string
        • os object
          Hide os attributes Show os attributes object
          • cpu object
            Hide cpu attributes Show cpu attributes object
            • percent number
            • sys string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • user string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • load_average object
          • swap object
            Hide swap attributes Show swap attributes object
            • adjusted_total_in_bytes number

              If the amount of physical memory has been overridden using the es.total_memory_bytes system property then this reports the overridden value in bytes. Otherwise it reports the same value as total_in_bytes.

            • resident string
            • resident_in_bytes number
            • share string
            • share_in_bytes number
            • total_virtual string
            • total_virtual_in_bytes number
            • total_in_bytes number

              Total amount of physical memory in bytes.

            • free_in_bytes number

              Amount of free physical memory in bytes.

            • used_in_bytes number

              Amount of used physical memory in bytes.

          • cgroup object
            Hide cgroup attributes Show cgroup attributes object
            • cpuacct object
            • cpu object
            • memory object
          • timestamp number
        • process object
          Hide process attributes Show process attributes object
          • cpu object
            Hide cpu attributes Show cpu attributes object
            • percent number
            • sys string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • user string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • load_average object
          • mem object
            Hide mem attributes Show mem attributes object
            • adjusted_total_in_bytes number

              If the amount of physical memory has been overridden using the es.total_memory_bytes system property then this reports the overridden value in bytes. Otherwise it reports the same value as total_in_bytes.

            • resident string
            • resident_in_bytes number
            • share string
            • share_in_bytes number
            • total_virtual string
            • total_virtual_in_bytes number
            • total_in_bytes number

              Total amount of physical memory in bytes.

            • free_in_bytes number

              Amount of free physical memory in bytes.

            • used_in_bytes number

              Amount of used physical memory in bytes.

          • open_file_descriptors number

            Number of opened file descriptors associated with the current or -1 if not supported.

          • max_file_descriptors number

            Maximum number of file descriptors allowed on the system, or -1 if not supported.

          • timestamp number

            Last time the statistics were refreshed. Recorded in milliseconds since the Unix Epoch.

        • roles array[string]

          Values are master, data, data_cold, data_content, data_frozen, data_hot, data_warm, client, ingest, ml, voting_only, transform, remote_cluster_client, or coordinating_only.

        • script object
          Hide script attributes Show script attributes object
          • cache_evictions number

            Total number of times the script cache has evicted old data.

          • compilations number

            Total number of inline script compilations performed by the node.

          • compilations_history object

            Contains this recent history of script compilations.

            Hide compilations_history attribute Show compilations_history attribute object
            • * number Additional properties
          • compilation_limit_triggered number

            Total number of times the script compilation circuit breaker has limited inline script compilations.

          • contexts array[object]
        • script_cache object
        • thread_pool object

          Statistics about each thread pool, including current size, queue and rejected tasks.

          Hide thread_pool attribute Show thread_pool attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • active number

              Number of active threads in the thread pool.

            • completed number

              Number of tasks completed by the thread pool executor.

            • largest number

              Highest number of active threads in the thread pool.

            • queue number

              Number of tasks in queue for the thread pool.

            • rejected number

              Number of tasks rejected by the thread pool executor.

            • threads number

              Number of threads in the thread pool.

        • timestamp number
        • transport object
          Hide transport attributes Show transport attributes object
          • inbound_handling_time_histogram array[object]

            The distribution of the time spent handling each inbound message on a transport thread, represented as a histogram.

          • outbound_handling_time_histogram array[object]

            The distribution of the time spent sending each outbound transport message on a transport thread, represented as a histogram.

          • rx_count number

            Total number of RX (receive) packets received by the node during internal cluster communication.

          • rx_size string

            Size of RX packets received by the node during internal cluster communication.

          • rx_size_in_bytes number

            Size, in bytes, of RX packets received by the node during internal cluster communication.

          • server_open number

            Current number of inbound TCP connections used for internal communication between nodes.

          • tx_count number

            Total number of TX (transmit) packets sent by the node during internal cluster communication.

          • tx_size string

            Size of TX packets sent by the node during internal cluster communication.

          • tx_size_in_bytes number

            Size, in bytes, of TX packets sent by the node during internal cluster communication.

          • total_outbound_connections number

            The cumulative number of outbound transport connections that this node has opened since it started. Each transport connection may comprise multiple TCP connections but is only counted once in this statistic. Transport connections are typically long-lived so this statistic should remain constant in a stable cluster.

        • transport_address string
        • attributes object

          Contains a list of attributes for the node.

          Hide attributes attribute Show attributes attribute object
          • * string Additional properties
        • discovery object
          Hide discovery attributes Show discovery attributes object
          • cluster_state_queue object
            Hide cluster_state_queue attributes Show cluster_state_queue attributes object
            • total number

              Total number of cluster states in queue.

            • pending number

              Number of pending cluster states in queue.

            • committed number

              Number of committed cluster states in queue.

          • published_cluster_states object
            Hide published_cluster_states attributes Show published_cluster_states attributes object
            • full_states number

              Number of published cluster states.

            • incompatible_diffs number

              Number of incompatible differences between published cluster states.

            • compatible_diffs number

              Number of compatible differences between published cluster states.

          • cluster_state_update object

            Contains low-level statistics about how long various activities took during cluster state updates while the node was the elected master. Omitted if the node is not master-eligible. Every field whose name ends in _time within this object is also represented as a raw number of milliseconds in a field whose name ends in _time_millis. The human-readable fields with a _time suffix are only returned if requested with the ?human=true query parameter.

            Hide cluster_state_update attribute Show cluster_state_update attribute object
            • * object Additional properties
          • serialized_cluster_states object
            Hide serialized_cluster_states attributes Show serialized_cluster_states attributes object
            • full_states object
            • diffs object
          • cluster_applier_stats object
            Hide cluster_applier_stats attribute Show cluster_applier_stats attribute object
            • recordings array[object]
        • indexing_pressure object
          Hide indexing_pressure attribute Show indexing_pressure attribute object
          • memory object
            Hide memory attributes Show memory attributes object
            • limit
            • limit_in_bytes number

              Configured memory limit, in bytes, for the indexing requests. Replica requests have an automatic limit that is 1.5x this value.

            • current object
            • total object
        • indices object
          Hide indices attributes Show indices attributes object
          • commit object
            Hide commit attributes Show commit attributes object
            • generation number Required
            • id string Required
            • num_docs number Required
            • user_data object Required
          • completion object
            Hide completion attributes Show completion attributes object
            • size_in_bytes number Required

              Total amount, in bytes, of memory used for completion across all shards assigned to selected nodes.

            • size
            • fields object
          • docs object
            Hide docs attributes Show docs attributes object
            • count number Required

              Total number of non-deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments and may include documents from nested fields.

            • deleted number

              Total number of deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments. Elasticsearch reclaims the disk space of deleted Lucene documents when a segment is merged.

          • fielddata object
            Hide fielddata attributes Show fielddata attributes object
            • evictions number
            • memory_size
            • memory_size_in_bytes number Required
            • fields object
          • flush object
            Hide flush attributes Show flush attributes object
            • periodic number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • get object
            Hide get attributes Show get attributes object
            • current number Required
            • exists_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • exists_total number Required
            • missing_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • missing_total number Required
            • time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total number Required
          • indexing object
            Hide indexing attributes Show indexing attributes object
            • index_current number Required
            • delete_current number Required
            • delete_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • delete_total number Required
            • is_throttled boolean Required
            • noop_update_total number Required
            • throttle_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • index_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • index_total number Required
            • index_failed number Required
            • types object
            • write_load number
          • mappings object
            Hide mappings attributes Show mappings attributes object
            • total_count number Required
            • total_estimated_overhead
            • total_estimated_overhead_in_bytes number Required
          • merges object
            Hide merges attributes Show merges attributes object
            • current number Required
            • current_docs number Required
            • current_size string
            • current_size_in_bytes number Required
            • total number Required
            • total_auto_throttle string
            • total_auto_throttle_in_bytes number Required
            • total_docs number Required
            • total_size string
            • total_size_in_bytes number Required
            • total_stopped_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_throttled_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • shard_path object
            Hide shard_path attributes Show shard_path attributes object
            • data_path string Required
            • is_custom_data_path boolean Required
            • state_path string Required
          • query_cache object
            Hide query_cache attributes Show query_cache attributes object
            • cache_count number Required
            • cache_size number Required
            • evictions number Required
            • hit_count number Required
            • memory_size_in_bytes number Required
            • miss_count number Required
            • total_count number Required
          • recovery object
            Hide recovery attributes Show recovery attributes object
            • current_as_source number Required
            • current_as_target number Required
            • throttle_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • refresh object
            Hide refresh attributes Show refresh attributes object
            • external_total number Required
            • listeners number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • request_cache object
            Hide request_cache attributes Show request_cache attributes object
            • evictions number Required
            • hit_count number Required
            • memory_size string
            • memory_size_in_bytes number Required
            • miss_count number Required
          • retention_leases object
            Hide retention_leases attributes Show retention_leases attributes object
            • primary_term number Required
            • version number Required
            • leases array[object] Required
          • routing object
            Hide routing attributes Show routing attributes object
            • node string Required
            • primary boolean Required
            • relocating_node
            • state string Required

              Values are UNASSIGNED, INITIALIZING, STARTED, or RELOCATING.

          • segments object
            Hide segments attributes Show segments attributes object
            • count number Required

              Total number of segments across all shards assigned to selected nodes.

            • doc_values_memory
            • doc_values_memory_in_bytes number Required

              Total amount, in bytes, of memory used for doc values across all shards assigned to selected nodes.

            • file_sizes object Required

              This object is not populated by the cluster stats API. To get information on segment files, use the node stats API.

            • fixed_bit_set
            • fixed_bit_set_memory_in_bytes number Required

              Total amount of memory, in bytes, used by fixed bit sets across all shards assigned to selected nodes.

            • index_writer_memory
            • index_writer_max_memory_in_bytes number
            • index_writer_memory_in_bytes number Required

              Total amount, in bytes, of memory used by all index writers across all shards assigned to selected nodes.

            • max_unsafe_auto_id_timestamp number Required

              Unix timestamp, in milliseconds, of the most recently retried indexing request.

            • memory
            • memory_in_bytes number Required

              Total amount, in bytes, of memory used for segments across all shards assigned to selected nodes.

            • norms_memory
            • norms_memory_in_bytes number Required

              Total amount, in bytes, of memory used for normalization factors across all shards assigned to selected nodes.

            • points_memory
            • points_memory_in_bytes number Required

              Total amount, in bytes, of memory used for points across all shards assigned to selected nodes.

            • stored_memory
            • stored_fields_memory_in_bytes number Required

              Total amount, in bytes, of memory used for stored fields across all shards assigned to selected nodes.

            • terms_memory_in_bytes number Required

              Total amount, in bytes, of memory used for terms across all shards assigned to selected nodes.

            • terms_memory
            • term_vectory_memory
            • term_vectors_memory_in_bytes number Required

              Total amount, in bytes, of memory used for term vectors across all shards assigned to selected nodes.

            • version_map_memory
            • version_map_memory_in_bytes number Required

              Total amount, in bytes, of memory used by all version maps across all shards assigned to selected nodes.

          • seq_no object
            Hide seq_no attributes Show seq_no attributes object
            • global_checkpoint number Required
            • local_checkpoint number Required
            • max_seq_no number Required
          • store object
            Hide store attributes Show store attributes object
            • size
            • size_in_bytes number Required

              Total size, in bytes, of all shards assigned to selected nodes.

            • reserved
            • reserved_in_bytes number Required

              A prediction, in bytes, of how much larger the shard stores will eventually grow due to ongoing peer recoveries, restoring snapshots, and similar activities.

            • total_data_set_size
            • total_data_set_size_in_bytes number

              Total data set size, in bytes, of all shards assigned to selected nodes. This includes the size of shards not stored fully on the nodes, such as the cache for partially mounted indices.

          • translog object
            Hide translog attributes Show translog attributes object
            • earliest_last_modified_age number Required
            • operations number Required
            • size string
            • size_in_bytes number Required
            • uncommitted_operations number Required
            • uncommitted_size string
            • uncommitted_size_in_bytes number Required
          • warmer object
            Hide warmer attributes Show warmer attributes object
            • current number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • bulk object
            Hide bulk attributes Show bulk attributes object
            • total_operations number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_size
            • total_size_in_bytes number Required
            • avg_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • avg_size
            • avg_size_in_bytes number Required
          • shards object Generally available; Added in 7.15.0
            Hide shards attribute Show shards attribute object
            • * object Additional properties
          • shard_stats object
            Hide shard_stats attribute Show shard_stats attribute object
            • total_count number Required
          • indices object Additional properties
            Hide indices attributes Show indices attributes object
            • primaries object
            • shards object
            • total object
            • uuid string
            • health string

              Values are green, GREEN, yellow, YELLOW, red, or RED.

            • status string

              Values are open or close.

GET /_nodes/stats
curl \
 --request GET 'http://api.example.com/_nodes/stats' \
 --header "Authorization: $API_KEY"
















Get node statistics Generally available

GET /_nodes/{node_id}/stats/{metric}/{index_metric}

Get statistics for nodes in a cluster. By default, all stats are returned. You can limit the returned information by using metrics.

Required authorization

  • Cluster privileges: monitor,manage

Path parameters

  • node_id string | array[string] Required

    Comma-separated list of node IDs or names used to limit returned information.

  • metric string | array[string] Required

    Limit the information returned to the specified metrics

  • index_metric string | array[string] Required

    Limit the information returned for indices metric to the specific index metrics. It can be used only if indices (or all) metric is specified.

Query parameters

  • completion_fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in fielddata and suggest statistics.

  • fielddata_fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in fielddata statistics.

  • fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in the statistics.

  • groups boolean

    Comma-separated list of search groups to include in the search statistics.

  • include_segment_file_sizes boolean

    If true, the call reports the aggregated disk usage of each one of the Lucene index files (only applies if segment stats are requested).

  • level string

    Indicates whether statistics are aggregated at the cluster, index, or shard level.

    Values are cluster, indices, or shards.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • types array[string]

    A comma-separated list of document types for the indexing index metric.

  • include_unloaded_segments boolean

    If true, the response includes information from segments that are not loaded into memory.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object

      Contains statistics about the number of nodes selected by the request.

      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide failures attributes Show failures attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string
    • nodes object Required
      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • adaptive_selection object

          Statistics about adaptive replica selection.

          Hide adaptive_selection attribute Show adaptive_selection attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • avg_queue_size number

              The exponentially weighted moving average queue size of search requests on the keyed node.

            • avg_response_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • avg_response_time_ns number

              The exponentially weighted moving average response time, in nanoseconds, of search requests on the keyed node.

            • avg_service_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • avg_service_time_ns number

              The exponentially weighted moving average service time, in nanoseconds, of search requests on the keyed node.

            • outgoing_searches number

              The number of outstanding search requests to the keyed node from the node these stats are for.

            • rank string

              The rank of this node; used for shard selection when routing search requests.

        • breakers object

          Statistics about the field data circuit breaker.

          Hide breakers attribute Show breakers attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • estimated_size string

              Estimated memory used for the operation.

            • estimated_size_in_bytes number

              Estimated memory used, in bytes, for the operation.

            • limit_size string

              Memory limit for the circuit breaker.

            • limit_size_in_bytes number

              Memory limit, in bytes, for the circuit breaker.

            • overhead number

              A constant that all estimates for the circuit breaker are multiplied with to calculate a final estimate.

            • tripped number

              Total number of times the circuit breaker has been triggered and prevented an out of memory error.

        • fs object
          Hide fs attributes Show fs attributes object
          • data array[object]

            List of all file stores.

          • timestamp number

            Last time the file stores statistics were refreshed. Recorded in milliseconds since the Unix Epoch.

          • total object
            Hide total attributes Show total attributes object
            • available string

              Total disk space available to this Java virtual machine on all file stores. Depending on OS or process level restrictions, this might appear less than free. This is the actual amount of free disk space the Elasticsearch node can utilise.

            • available_in_bytes number

              Total number of bytes available to this Java virtual machine on all file stores. Depending on OS or process level restrictions, this might appear less than free_in_bytes. This is the actual amount of free disk space the Elasticsearch node can utilise.

            • free string

              Total unallocated disk space in all file stores.

            • free_in_bytes number

              Total number of unallocated bytes in all file stores.

            • total string

              Total size of all file stores.

            • total_in_bytes number

              Total size of all file stores in bytes.

          • io_stats object
            Hide io_stats attributes Show io_stats attributes object
            • devices array[object]

              Array of disk metrics for each device that is backing an Elasticsearch data path. These disk metrics are probed periodically and averages between the last probe and the current probe are computed.

            • total object
        • host string
        • http object
          Hide http attributes Show http attributes object
          • current_open number

            Current number of open HTTP connections for the node.

          • total_opened number

            Total number of HTTP connections opened for the node.

          • clients array[object]

            Information on current and recently-closed HTTP client connections. Clients that have been closed longer than the http.client_stats.closed_channels.max_age setting will not be represented here.

          • routes object Required Generally available; Added in 8.12.0

            Detailed HTTP stats broken down by route

            Hide routes attribute Show routes attribute object
            • * object Additional properties
        • ingest object
          Hide ingest attributes Show ingest attributes object
          • pipelines object

            Contains statistics about ingest pipelines for the node.

            Hide pipelines attribute Show pipelines attribute object
            • * object Additional properties
          • total object
            Hide total attributes Show total attributes object
            • count number Required

              Total number of documents ingested during the lifetime of this node.

            • current number Required

              Total number of documents currently being ingested.

            • failed number Required

              Total number of failed ingest operations during the lifetime of this node.

        • ip string | array[string]

          IP address and port for the node.

        • jvm object
          Hide jvm attributes Show jvm attributes object
          • buffer_pools object

            Contains statistics about JVM buffer pools for the node.

            Hide buffer_pools attribute Show buffer_pools attribute object
            • * object Additional properties
          • classes object
            Hide classes attributes Show classes attributes object
            • current_loaded_count number

              Number of classes currently loaded by JVM.

            • total_loaded_count number

              Total number of classes loaded since the JVM started.

            • total_unloaded_count number

              Total number of classes unloaded since the JVM started.

          • gc object
            Hide gc attribute Show gc attribute object
            • collectors object

              Contains statistics about JVM garbage collectors for the node.

          • mem object
            Hide mem attributes Show mem attributes object
            • heap_used_in_bytes number

              Memory, in bytes, currently in use by the heap.

            • heap_used_percent number

              Percentage of memory currently in use by the heap.

            • heap_committed_in_bytes number

              Amount of memory, in bytes, available for use by the heap.

            • heap_max_in_bytes number

              Maximum amount of memory, in bytes, available for use by the heap.

            • non_heap_used_in_bytes number

              Non-heap memory used, in bytes.

            • non_heap_committed_in_bytes number

              Amount of non-heap memory available, in bytes.

            • pools object

              Contains statistics about heap memory usage for the node.

          • threads object
            Hide threads attributes Show threads attributes object
            • count number

              Number of active threads in use by JVM.

            • peak_count number

              Highest number of threads used by JVM.

          • timestamp number

            Last time JVM statistics were refreshed.

          • uptime string

            Human-readable JVM uptime. Only returned if the human query parameter is true.

          • uptime_in_millis number

            JVM uptime in milliseconds.

        • name string
        • os object
          Hide os attributes Show os attributes object
          • cpu object
            Hide cpu attributes Show cpu attributes object
            • percent number
            • sys string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • user string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • load_average object
          • swap object
            Hide swap attributes Show swap attributes object
            • adjusted_total_in_bytes number

              If the amount of physical memory has been overridden using the es.total_memory_bytes system property then this reports the overridden value in bytes. Otherwise it reports the same value as total_in_bytes.

            • resident string
            • resident_in_bytes number
            • share string
            • share_in_bytes number
            • total_virtual string
            • total_virtual_in_bytes number
            • total_in_bytes number

              Total amount of physical memory in bytes.

            • free_in_bytes number

              Amount of free physical memory in bytes.

            • used_in_bytes number

              Amount of used physical memory in bytes.

          • cgroup object
            Hide cgroup attributes Show cgroup attributes object
            • cpuacct object
            • cpu object
            • memory object
          • timestamp number
        • process object
          Hide process attributes Show process attributes object
          • cpu object
            Hide cpu attributes Show cpu attributes object
            • percent number
            • sys string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • user string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • load_average object
          • mem object
            Hide mem attributes Show mem attributes object
            • adjusted_total_in_bytes number

              If the amount of physical memory has been overridden using the es.total_memory_bytes system property then this reports the overridden value in bytes. Otherwise it reports the same value as total_in_bytes.

            • resident string
            • resident_in_bytes number
            • share string
            • share_in_bytes number
            • total_virtual string
            • total_virtual_in_bytes number
            • total_in_bytes number

              Total amount of physical memory in bytes.

            • free_in_bytes number

              Amount of free physical memory in bytes.

            • used_in_bytes number

              Amount of used physical memory in bytes.

          • open_file_descriptors number

            Number of opened file descriptors associated with the current or -1 if not supported.

          • max_file_descriptors number

            Maximum number of file descriptors allowed on the system, or -1 if not supported.

          • timestamp number

            Last time the statistics were refreshed. Recorded in milliseconds since the Unix Epoch.

        • roles array[string]

          Values are master, data, data_cold, data_content, data_frozen, data_hot, data_warm, client, ingest, ml, voting_only, transform, remote_cluster_client, or coordinating_only.

        • script object
          Hide script attributes Show script attributes object
          • cache_evictions number

            Total number of times the script cache has evicted old data.

          • compilations number

            Total number of inline script compilations performed by the node.

          • compilations_history object

            Contains this recent history of script compilations.

            Hide compilations_history attribute Show compilations_history attribute object
            • * number Additional properties
          • compilation_limit_triggered number

            Total number of times the script compilation circuit breaker has limited inline script compilations.

          • contexts array[object]
        • script_cache object
        • thread_pool object

          Statistics about each thread pool, including current size, queue and rejected tasks.

          Hide thread_pool attribute Show thread_pool attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • active number

              Number of active threads in the thread pool.

            • completed number

              Number of tasks completed by the thread pool executor.

            • largest number

              Highest number of active threads in the thread pool.

            • queue number

              Number of tasks in queue for the thread pool.

            • rejected number

              Number of tasks rejected by the thread pool executor.

            • threads number

              Number of threads in the thread pool.

        • timestamp number
        • transport object
          Hide transport attributes Show transport attributes object
          • inbound_handling_time_histogram array[object]

            The distribution of the time spent handling each inbound message on a transport thread, represented as a histogram.

          • outbound_handling_time_histogram array[object]

            The distribution of the time spent sending each outbound transport message on a transport thread, represented as a histogram.

          • rx_count number

            Total number of RX (receive) packets received by the node during internal cluster communication.

          • rx_size string

            Size of RX packets received by the node during internal cluster communication.

          • rx_size_in_bytes number

            Size, in bytes, of RX packets received by the node during internal cluster communication.

          • server_open number

            Current number of inbound TCP connections used for internal communication between nodes.

          • tx_count number

            Total number of TX (transmit) packets sent by the node during internal cluster communication.

          • tx_size string

            Size of TX packets sent by the node during internal cluster communication.

          • tx_size_in_bytes number

            Size, in bytes, of TX packets sent by the node during internal cluster communication.

          • total_outbound_connections number

            The cumulative number of outbound transport connections that this node has opened since it started. Each transport connection may comprise multiple TCP connections but is only counted once in this statistic. Transport connections are typically long-lived so this statistic should remain constant in a stable cluster.

        • transport_address string
        • attributes object

          Contains a list of attributes for the node.

          Hide attributes attribute Show attributes attribute object
          • * string Additional properties
        • discovery object
          Hide discovery attributes Show discovery attributes object
          • cluster_state_queue object
            Hide cluster_state_queue attributes Show cluster_state_queue attributes object
            • total number

              Total number of cluster states in queue.

            • pending number

              Number of pending cluster states in queue.

            • committed number

              Number of committed cluster states in queue.

          • published_cluster_states object
            Hide published_cluster_states attributes Show published_cluster_states attributes object
            • full_states number

              Number of published cluster states.

            • incompatible_diffs number

              Number of incompatible differences between published cluster states.

            • compatible_diffs number

              Number of compatible differences between published cluster states.

          • cluster_state_update object

            Contains low-level statistics about how long various activities took during cluster state updates while the node was the elected master. Omitted if the node is not master-eligible. Every field whose name ends in _time within this object is also represented as a raw number of milliseconds in a field whose name ends in _time_millis. The human-readable fields with a _time suffix are only returned if requested with the ?human=true query parameter.

            Hide cluster_state_update attribute Show cluster_state_update attribute object
            • * object Additional properties
          • serialized_cluster_states object
            Hide serialized_cluster_states attributes Show serialized_cluster_states attributes object
            • full_states object
            • diffs object
          • cluster_applier_stats object
            Hide cluster_applier_stats attribute Show cluster_applier_stats attribute object
            • recordings array[object]
        • indexing_pressure object
          Hide indexing_pressure attribute Show indexing_pressure attribute object
          • memory object
            Hide memory attributes Show memory attributes object
            • limit
            • limit_in_bytes number

              Configured memory limit, in bytes, for the indexing requests. Replica requests have an automatic limit that is 1.5x this value.

            • current object
            • total object
        • indices object
          Hide indices attributes Show indices attributes object
          • commit object
            Hide commit attributes Show commit attributes object
            • generation number Required
            • id string Required
            • num_docs number Required
            • user_data object Required
          • completion object
            Hide completion attributes Show completion attributes object
            • size_in_bytes number Required

              Total amount, in bytes, of memory used for completion across all shards assigned to selected nodes.

            • size
            • fields object
          • docs object
            Hide docs attributes Show docs attributes object
            • count number Required

              Total number of non-deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments and may include documents from nested fields.

            • deleted number

              Total number of deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments. Elasticsearch reclaims the disk space of deleted Lucene documents when a segment is merged.

          • fielddata object
            Hide fielddata attributes Show fielddata attributes object
            • evictions number
            • memory_size
            • memory_size_in_bytes number Required
            • fields object
          • flush object
            Hide flush attributes Show flush attributes object
            • periodic number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • get object
            Hide get attributes Show get attributes object
            • current number Required
            • exists_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • exists_total number Required
            • missing_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • missing_total number Required
            • time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total number Required
          • indexing object
            Hide indexing attributes Show indexing attributes object
            • index_current number Required
            • delete_current number Required
            • delete_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • delete_total number Required
            • is_throttled boolean Required
            • noop_update_total number Required
            • throttle_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • index_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • index_total number Required
            • index_failed number Required
            • types object
            • write_load number
          • mappings object
            Hide mappings attributes Show mappings attributes object
            • total_count number Required
            • total_estimated_overhead
            • total_estimated_overhead_in_bytes number Required
          • merges object
            Hide merges attributes Show merges attributes object
            • current number Required
            • current_docs number Required
            • current_size string
            • current_size_in_bytes number Required
            • total number Required
            • total_auto_throttle string
            • total_auto_throttle_in_bytes number Required
            • total_docs number Required
            • total_size string
            • total_size_in_bytes number Required
            • total_stopped_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_throttled_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • shard_path object
            Hide shard_path attributes Show shard_path attributes object
            • data_path string Required
            • is_custom_data_path boolean Required
            • state_path string Required
          • query_cache object
            Hide query_cache attributes Show query_cache attributes object
            • cache_count number Required
            • cache_size number Required
            • evictions number Required
            • hit_count number Required
            • memory_size_in_bytes number Required
            • miss_count number Required
            • total_count number Required
          • recovery object
            Hide recovery attributes Show recovery attributes object
            • current_as_source number Required
            • current_as_target number Required
            • throttle_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • refresh object
            Hide refresh attributes Show refresh attributes object
            • external_total number Required
            • listeners number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • request_cache object
            Hide request_cache attributes Show request_cache attributes object
            • evictions number Required
            • hit_count number Required
            • memory_size string
            • memory_size_in_bytes number Required
            • miss_count number Required
          • retention_leases object
            Hide retention_leases attributes Show retention_leases attributes object
            • primary_term number Required
            • version number Required
            • leases array[object] Required
          • routing object
            Hide routing attributes Show routing attributes object
            • node string Required
            • primary boolean Required
            • relocating_node
            • state string Required

              Values are UNASSIGNED, INITIALIZING, STARTED, or RELOCATING.

          • segments object
            Hide segments attributes Show segments attributes object
            • count number Required

              Total number of segments across all shards assigned to selected nodes.

            • doc_values_memory
            • doc_values_memory_in_bytes number Required

              Total amount, in bytes, of memory used for doc values across all shards assigned to selected nodes.

            • file_sizes object Required

              This object is not populated by the cluster stats API. To get information on segment files, use the node stats API.

            • fixed_bit_set
            • fixed_bit_set_memory_in_bytes number Required

              Total amount of memory, in bytes, used by fixed bit sets across all shards assigned to selected nodes.

            • index_writer_memory
            • index_writer_max_memory_in_bytes number
            • index_writer_memory_in_bytes number Required

              Total amount, in bytes, of memory used by all index writers across all shards assigned to selected nodes.

            • max_unsafe_auto_id_timestamp number Required

              Unix timestamp, in milliseconds, of the most recently retried indexing request.

            • memory
            • memory_in_bytes number Required

              Total amount, in bytes, of memory used for segments across all shards assigned to selected nodes.

            • norms_memory
            • norms_memory_in_bytes number Required

              Total amount, in bytes, of memory used for normalization factors across all shards assigned to selected nodes.

            • points_memory
            • points_memory_in_bytes number Required

              Total amount, in bytes, of memory used for points across all shards assigned to selected nodes.

            • stored_memory
            • stored_fields_memory_in_bytes number Required

              Total amount, in bytes, of memory used for stored fields across all shards assigned to selected nodes.

            • terms_memory_in_bytes number Required

              Total amount, in bytes, of memory used for terms across all shards assigned to selected nodes.

            • terms_memory
            • term_vectory_memory
            • term_vectors_memory_in_bytes number Required

              Total amount, in bytes, of memory used for term vectors across all shards assigned to selected nodes.

            • version_map_memory
            • version_map_memory_in_bytes number Required

              Total amount, in bytes, of memory used by all version maps across all shards assigned to selected nodes.

          • seq_no object
            Hide seq_no attributes Show seq_no attributes object
            • global_checkpoint number Required
            • local_checkpoint number Required
            • max_seq_no number Required
          • store object
            Hide store attributes Show store attributes object
            • size
            • size_in_bytes number Required

              Total size, in bytes, of all shards assigned to selected nodes.

            • reserved
            • reserved_in_bytes number Required

              A prediction, in bytes, of how much larger the shard stores will eventually grow due to ongoing peer recoveries, restoring snapshots, and similar activities.

            • total_data_set_size
            • total_data_set_size_in_bytes number

              Total data set size, in bytes, of all shards assigned to selected nodes. This includes the size of shards not stored fully on the nodes, such as the cache for partially mounted indices.

          • translog object
            Hide translog attributes Show translog attributes object
            • earliest_last_modified_age number Required
            • operations number Required
            • size string
            • size_in_bytes number Required
            • uncommitted_operations number Required
            • uncommitted_size string
            • uncommitted_size_in_bytes number Required
          • warmer object
            Hide warmer attributes Show warmer attributes object
            • current number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • bulk object
            Hide bulk attributes Show bulk attributes object
            • total_operations number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_size
            • total_size_in_bytes number Required
            • avg_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • avg_size
            • avg_size_in_bytes number Required
          • shards object Generally available; Added in 7.15.0
            Hide shards attribute Show shards attribute object
            • * object Additional properties
          • shard_stats object
            Hide shard_stats attribute Show shard_stats attribute object
            • total_count number Required
          • indices object Additional properties
            Hide indices attributes Show indices attributes object
            • primaries object
            • shards object
            • total object
            • uuid string
            • health string

              Values are green, GREEN, yellow, YELLOW, red, or RED.

            • status string

              Values are open or close.

GET /_nodes/{node_id}/stats/{metric}/{index_metric}
curl \
 --request GET 'http://api.example.com/_nodes/{node_id}/stats/{metric}/{index_metric}' \
 --header "Authorization: $API_KEY"

Get feature usage information Generally available; Added in 6.0.0

GET /_nodes/usage

Required authorization

  • Cluster privileges: monitor,manage

Query parameters

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object

      Contains statistics about the number of nodes selected by the request.

      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide failures attributes Show failures attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • nodes object Required
      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • rest_actions object Required
          Hide rest_actions attribute Show rest_actions attribute object
          • * number Additional properties
        • Time unit for milliseconds

        • Time unit for milliseconds

        • aggregations object Required
          Hide aggregations attribute Show aggregations attribute object
          • * object Additional properties
GET /_nodes/usage
curl \
 --request GET 'http://api.example.com/_nodes/usage' \
 --header "Authorization: $API_KEY"

Get feature usage information Generally available; Added in 6.0.0

GET /_nodes/{node_id}/usage

Required authorization

  • Cluster privileges: monitor,manage

Path parameters

  • node_id string | array[string] Required

    A comma-separated list of node IDs or names to limit the returned information; use _local to return information from the node you're connecting to, leave empty to get information from all nodes

Query parameters

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object

      Contains statistics about the number of nodes selected by the request.

      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide failures attributes Show failures attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • nodes object Required
      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • rest_actions object Required
          Hide rest_actions attribute Show rest_actions attribute object
          • * number Additional properties
        • Time unit for milliseconds

        • Time unit for milliseconds

        • aggregations object Required
          Hide aggregations attribute Show aggregations attribute object
          • * object Additional properties
GET /_nodes/{node_id}/usage
curl \
 --request GET 'http://api.example.com/_nodes/{node_id}/usage' \
 --header "Authorization: $API_KEY"








Cluster - Health

Get the cluster health Generally available; Added in 8.7.0

GET /_health_report

Get a report with the health status of an Elasticsearch cluster. The report contains a list of indicators that compose Elasticsearch functionality.

Each indicator has a health status of: green, unknown, yellow or red. The indicator will provide an explanation and metadata describing the reason for its current health status.

The cluster’s status is controlled by the worst indicator status.

In the event that an indicator’s status is non-green, a list of impacts may be present in the indicator result which detail the functionalities that are negatively affected by the health issue. Each impact carries with it a severity level, an area of the system that is affected, and a simple description of the impact on the system.

Some health indicators can determine the root cause of a health problem and prescribe a set of steps that can be performed in order to improve the health of the system. The root cause and remediation steps are encapsulated in a diagnosis. A diagnosis contains a cause detailing a root cause analysis, an action containing a brief description of the steps to take to fix the problem, the list of affected resources (if applicable), and a detailed step-by-step troubleshooting guide to fix the diagnosed problem.

NOTE: The health indicators perform root cause analysis of non-green health statuses. This can be computationally expensive when called frequently. When setting up automated polling of the API for health status, set verbose to false to disable the more expensive analysis logic.

Query parameters

  • timeout string

    Explicit operation timeout.

    Values are -1 or 0.

  • verbose boolean

    Opt-in for more information about the health of the system.

  • size number

    Limit the number of affected resources the health report API returns.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • cluster_name string Required
    • indicators object Required
      Hide indicators attributes Show indicators attributes object
      • master_is_stable object

        MASTER_IS_STABLE

        Hide master_is_stable attributes Show master_is_stable attributes object
        • status string Required

          Values are green, yellow, red, or unknown.

        • symptom string Required
        • impacts array[object]
          Hide impacts attributes Show impacts attributes object
          • description string Required
          • id string Required
          • impact_areas array[string] Required

            Values are search, ingest, backup, or deployment_management.

          • severity number Required
        • diagnosis array[object]
          Hide diagnosis attributes Show diagnosis attributes object
          • id string Required
          • action string Required
          • affected_resources object Required
          • cause string Required
          • help_url string Required
        • details object
          Hide details attributes Show details attributes object
          • current_master object Required
            Hide current_master attributes Show current_master attributes object
            • name
            • node_id
          • recent_masters array[object] Required
          • exception_fetching_history object
            Hide exception_fetching_history attributes Show exception_fetching_history attributes object
            • message string Required
            • stack_trace string Required
          • cluster_formation array[object]
      • shards_availability object

        SHARDS_AVAILABILITY

        Hide shards_availability attributes Show shards_availability attributes object
        • status string Required

          Values are green, yellow, red, or unknown.

        • symptom string Required
        • impacts array[object]
          Hide impacts attributes Show impacts attributes object
          • description string Required
          • id string Required
          • impact_areas array[string] Required

            Values are search, ingest, backup, or deployment_management.

          • severity number Required
        • diagnosis array[object]
          Hide diagnosis attributes Show diagnosis attributes object
          • id string Required
          • action string Required
          • affected_resources object Required
          • cause string Required
          • help_url string Required
        • details object
          Hide details attributes Show details attributes object
          • creating_primaries number Required
          • creating_replicas number Required
          • initializing_primaries number Required
          • initializing_replicas number Required
          • restarting_primaries number Required
          • restarting_replicas number Required
          • started_primaries number Required
          • started_replicas number Required
          • unassigned_primaries number Required
          • unassigned_replicas number Required
      • disk object

        DISK

        Hide disk attributes Show disk attributes object
        • status string Required

          Values are green, yellow, red, or unknown.

        • symptom string Required
        • impacts array[object]
          Hide impacts attributes Show impacts attributes object
          • description string Required
          • id string Required
          • impact_areas array[string] Required

            Values are search, ingest, backup, or deployment_management.

          • severity number Required
        • diagnosis array[object]
          Hide diagnosis attributes Show diagnosis attributes object
          • id string Required
          • action string Required
          • affected_resources object Required
          • cause string Required
          • help_url string Required
        • details object
          Hide details attributes Show details attributes object
          • indices_with_readonly_block number Required
          • nodes_with_enough_disk_space number Required
          • nodes_over_high_watermark number Required
          • nodes_over_flood_stage_watermark number Required
          • nodes_with_unknown_disk_status number Required
      • repository_integrity object

        REPOSITORY_INTEGRITY

        Hide repository_integrity attributes Show repository_integrity attributes object
        • status string Required

          Values are green, yellow, red, or unknown.

        • symptom string Required
        • impacts array[object]
          Hide impacts attributes Show impacts attributes object
          • description string Required
          • id string Required
          • impact_areas array[string] Required

            Values are search, ingest, backup, or deployment_management.

          • severity number Required
        • diagnosis array[object]
          Hide diagnosis attributes Show diagnosis attributes object
          • id string Required
          • action string Required
          • affected_resources object Required
          • cause string Required
          • help_url string Required
        • details object
          Hide details attributes Show details attributes object
          • total_repositories number
          • corrupted_repositories number
          • corrupted array[string]
      • data_stream_lifecycle object

        DATA_STREAM_LIFECYCLE

        Hide data_stream_lifecycle attributes Show data_stream_lifecycle attributes object
        • status string Required

          Values are green, yellow, red, or unknown.

        • symptom string Required
        • impacts array[object]
          Hide impacts attributes Show impacts attributes object
          • description string Required
          • id string Required
          • impact_areas array[string] Required

            Values are search, ingest, backup, or deployment_management.

          • severity number Required
        • diagnosis array[object]
          Hide diagnosis attributes Show diagnosis attributes object
          • id string Required
          • action string Required
          • affected_resources object Required
          • cause string Required
          • help_url string Required
        • details object
          Hide details attributes Show details attributes object
          • stagnating_backing_indices_count number Required
          • total_backing_indices_in_error number Required
          • stagnating_backing_indices array[object]
      • ilm object

        ILM

        Hide ilm attributes Show ilm attributes object
        • status string Required

          Values are green, yellow, red, or unknown.

        • symptom string Required
        • impacts array[object]
          Hide impacts attributes Show impacts attributes object
          • description string Required
          • id string Required
          • impact_areas array[string] Required

            Values are search, ingest, backup, or deployment_management.

          • severity number Required
        • diagnosis array[object]
          Hide diagnosis attributes Show diagnosis attributes object
          • id string Required
          • action string Required
          • affected_resources object Required
          • cause string Required
          • help_url string Required
        • details object
          Hide details attributes Show details attributes object
          • ilm_status string Required

            Values are RUNNING, STOPPING, or STOPPED.

          • policies number Required
          • stagnating_indices number Required
      • slm object

        SLM

        Hide slm attributes Show slm attributes object
        • status string Required

          Values are green, yellow, red, or unknown.

        • symptom string Required
        • impacts array[object]
          Hide impacts attributes Show impacts attributes object
          • description string Required
          • id string Required
          • impact_areas array[string] Required

            Values are search, ingest, backup, or deployment_management.

          • severity number Required
        • diagnosis array[object]
          Hide diagnosis attributes Show diagnosis attributes object
          • id string Required
          • action string Required
          • affected_resources object Required
          • cause string Required
          • help_url string Required
        • details object
          Hide details attributes Show details attributes object
          • slm_status string Required

            Values are RUNNING, STOPPING, or STOPPED.

          • policies number Required
          • unhealthy_policies object
            Hide unhealthy_policies attributes Show unhealthy_policies attributes object
            • count number Required
            • invocations_since_last_success object
      • shards_capacity object

        SHARDS_CAPACITY

        Hide shards_capacity attributes Show shards_capacity attributes object
        • status string Required

          Values are green, yellow, red, or unknown.

        • symptom string Required
        • impacts array[object]
          Hide impacts attributes Show impacts attributes object
          • description string Required
          • id string Required
          • impact_areas array[string] Required

            Values are search, ingest, backup, or deployment_management.

          • severity number Required
        • diagnosis array[object]
          Hide diagnosis attributes Show diagnosis attributes object
          • id string Required
          • action string Required
          • affected_resources object Required
          • cause string Required
          • help_url string Required
        • details object
          Hide details attributes Show details attributes object
          • data object Required
            Hide data attributes Show data attributes object
            • max_shards_in_cluster number Required
            • current_used_shards number
          • frozen object Required
            Hide frozen attributes Show frozen attributes object
            • max_shards_in_cluster number Required
            • current_used_shards number
    • status string

      Values are green, yellow, red, or unknown.

GET /_health_report
curl \
 --request GET 'http://api.example.com/_health_report' \
 --header "Authorization: $API_KEY"




Connector

The connector and sync jobs APIs provide a convenient way to create and manage Elastic connectors and sync jobs in an internal index. Connectors are Elasticsearch integrations that bring content from third-party data sources, which can be deployed on Elastic Cloud or hosted on your own infrastructure:

  • Elastic managed connectors (Native connectors) are a managed service on Elastic Cloud
  • Self-managed connectors (Connector clients) are self-managed on your infrastructure.

This API provides an alternative to relying solely on Kibana UI for connector and sync job management. The API comes with a set of validations and assertions to ensure that the state representation in the internal index remains valid.

Check out the connector API tutorial

Check in a connector Technical preview; Added in 8.12.0

PUT /_connector/{connector_id}/_check_in

Update the last_seen field in the connector and set it to the current timestamp.

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be checked in

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_check_in
curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_check_in' \
 --header "Authorization: $API_KEY"
Response examples (200)
{
    "result": "updated"
}












































Set a connector sync job error Technical preview

PUT /_connector/_sync_job/{connector_sync_job_id}/_error

Set the error field for a connector sync job and set its status to error.

To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for Elastic managed connectors.

Path parameters

  • connector_sync_job_id string Required

    The unique identifier for the connector sync job.

application/json

Body Required

  • error string Required

    The error for the connector sync job error field.

Responses

  • 200 application/json
PUT /_connector/_sync_job/{connector_sync_job_id}/_error
curl \
 --request PUT 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}/_error' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"error\": \"some-error\"\n}"'
Request example
{
    "error": "some-error"
}












Activate the connector draft filter Technical preview; Added in 8.12.0

PUT /_connector/{connector_id}/_filtering/_activate

Activates the valid draft filtering for a connector.

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_filtering/_activate
curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_filtering/_activate' \
 --header "Authorization: $API_KEY"

Update the connector API key ID Beta; Added in 8.12.0

PUT /_connector/{connector_id}/_api_key_id

Update the api_key_id and api_key_secret_id fields of a connector. You can specify the ID of the API key used for authorization and the ID of the connector secret where the API key is stored. The connector secret ID is required only for Elastic managed (native) connectors. Self-managed connectors (connector clients) do not use this field.

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

application/json

Body Required

  • api_key_id string
  • api_key_secret_id string

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_api_key_id
curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_api_key_id' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"api_key_id\": \"my-api-key-id\",\n    \"api_key_secret_id\": \"my-connector-secret-id\"\n}"'
Request example
{
    "api_key_id": "my-api-key-id",
    "api_key_secret_id": "my-connector-secret-id"
}
Response examples (200)
{
  "result": "updated"
}




Update the connector error field Technical preview; Added in 8.12.0

PUT /_connector/{connector_id}/_error

Set the error field for the connector. If the error provided in the request body is non-null, the connector’s status is updated to error. Otherwise, if the error is reset to null, the connector status is updated to connected.

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

application/json

Body Required

  • error string | null Required

    One of:

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_error
curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_error' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"error\": \"Houston, we have a problem!\"\n}"'
Request example
{
    "error": "Houston, we have a problem!"
}
Response examples (200)
{
  "result": "updated"
}

Update the connector features Technical preview

PUT /_connector/{connector_id}/_features

Update the connector features in the connector document. This API can be used to control the following aspects of a connector:

  • document-level security
  • incremental syncs
  • advanced sync rules
  • basic sync rules

Normally, the running connector service automatically manages these features. However, you can use this API to override the default behavior.

To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for Elastic managed connectors.

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated.

application/json

Body Required

  • features object Required
    Hide features attributes Show features attributes object
    • document_level_security object
      Hide document_level_security attribute Show document_level_security attribute object
      • enabled boolean Required
    • incremental_sync object
      Hide incremental_sync attribute Show incremental_sync attribute object
      • enabled boolean Required
    • native_connector_api_keys object
      Hide native_connector_api_keys attribute Show native_connector_api_keys attribute object
      • enabled boolean Required
    • sync_rules object
      Hide sync_rules attributes Show sync_rules attributes object
      • advanced object
        Hide advanced attribute Show advanced attribute object
        • enabled boolean Required
      • basic object
        Hide basic attribute Show basic attribute object
        • enabled boolean Required

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_features
curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_features' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"features\": {\n    \"document_level_security\": {\n      \"enabled\": true\n    },\n    \"incremental_sync\": {\n      \"enabled\": true\n    },\n    \"sync_rules\": {\n      \"advanced\": {\n        \"enabled\": false\n      },\n      \"basic\": {\n        \"enabled\": true\n      }\n    }\n  }\n}"'
Request examples
{
  "features": {
    "document_level_security": {
      "enabled": true
    },
    "incremental_sync": {
      "enabled": true
    },
    "sync_rules": {
      "advanced": {
        "enabled": false
      },
      "basic": {
        "enabled": true
      }
    }
  }
}
{
  "features": {
    "document_level_security": {
      "enabled": true
    }
  }
}
Response examples (200)
{
  "result": "updated"
}

Update the connector filtering Beta; Added in 8.12.0

PUT /_connector/{connector_id}/_filtering

Update the draft filtering configuration of a connector and marks the draft validation state as edited. The filtering draft is activated once validated by the running Elastic connector service. The filtering property is used to configure sync rules (both basic and advanced) for a connector.

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

application/json

Body Required

  • filtering array[object]
    Hide filtering attributes Show filtering attributes object
    • active object Required
      Hide active attributes Show active attributes object
      • advanced_snippet object Required
        Hide advanced_snippet attributes Show advanced_snippet attributes object
        • created_at string | number

          A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

          One of:
        • updated_at string | number

          A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

          One of:
        • value object Required
      • rules array[object] Required
        Hide rules attributes Show rules attributes object
        • created_at string
        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • id string Required
        • order number Required
        • policy string Required

          Values are exclude or include.

        • rule string Required

          Values are contains, ends_with, equals, regex, starts_with, >, or <.

        • updated_at string
        • value string Required
      • validation object Required
        Hide validation attributes Show validation attributes object
        • errors array[object] Required
          Hide errors attributes Show errors attributes object
          • ids array[string] Required
          • messages array[string] Required
        • state string Required

          Values are edited, invalid, or valid.

    • domain string
    • draft object Required
      Hide draft attributes Show draft attributes object
      • advanced_snippet object Required
        Hide advanced_snippet attributes Show advanced_snippet attributes object
        • created_at string | number

          A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

          One of:
        • updated_at string | number

          A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

          One of:
        • value object Required
      • rules array[object] Required
        Hide rules attributes Show rules attributes object
        • created_at string
        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • id string Required
        • order number Required
        • policy string Required

          Values are exclude or include.

        • rule string Required

          Values are contains, ends_with, equals, regex, starts_with, >, or <.

        • updated_at string
        • value string Required
      • validation object Required
        Hide validation attributes Show validation attributes object
        • errors array[object] Required
          Hide errors attributes Show errors attributes object
          • ids array[string] Required
          • messages array[string] Required
        • state string Required

          Values are edited, invalid, or valid.

  • rules array[object]
    Hide rules attributes Show rules attributes object
    • created_at string | number

      A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

      One of:
    • field string Required

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • id string Required
    • order number Required
    • policy string Required

      Values are exclude or include.

    • rule string Required

      Values are contains, ends_with, equals, regex, starts_with, >, or <.

    • updated_at string | number

      A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

      One of:
    • value string Required
  • advanced_snippet object
    Hide advanced_snippet attributes Show advanced_snippet attributes object
    • created_at string | number

      A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

      One of:
    • updated_at string | number

      A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

      One of:
    • value object Required

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_filtering
curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_filtering' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"rules\": [\n         {\n            \"field\": \"file_extension\",\n            \"id\": \"exclude-txt-files\",\n            \"order\": 0,\n            \"policy\": \"exclude\",\n            \"rule\": \"equals\",\n            \"value\": \"txt\"\n        },\n        {\n            \"field\": \"_\",\n            \"id\": \"DEFAULT\",\n            \"order\": 1,\n            \"policy\": \"include\",\n            \"rule\": \"regex\",\n            \"value\": \".*\"\n        }\n    ]\n}"'
Request examples
{
    "rules": [
         {
            "field": "file_extension",
            "id": "exclude-txt-files",
            "order": 0,
            "policy": "exclude",
            "rule": "equals",
            "value": "txt"
        },
        {
            "field": "_",
            "id": "DEFAULT",
            "order": 1,
            "policy": "include",
            "rule": "regex",
            "value": ".*"
        }
    ]
}
{
    "advanced_snippet": {
        "value": [{
            "tables": [
                "users",
                "orders"
            ],
            "query": "SELECT users.id AS id, orders.order_id AS order_id FROM users JOIN orders ON users.id = orders.user_id"
        }]
    }
}
Response examples (200)
{
  "result": "updated"
}








Update the connector name and description Beta; Added in 8.12.0

PUT /_connector/{connector_id}/_name

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

application/json

Body Required

  • name string
  • description string

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_name
curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_name' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"name\": \"Custom connector\",\n    \"description\": \"This is my customized connector\"\n}"'
Request example
{
    "name": "Custom connector",
    "description": "This is my customized connector"
}
Response examples (200)
{
  "result": "updated"
}








Update the connector scheduling Beta; Added in 8.12.0

PUT /_connector/{connector_id}/_scheduling

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

application/json

Body Required

  • scheduling object Required
    Hide scheduling attributes Show scheduling attributes object
    • access_control object
      Hide access_control attributes Show access_control attributes object
      • enabled boolean Required
      • interval string Required

        The interval is expressed using the crontab syntax

    • full object
      Hide full attributes Show full attributes object
      • enabled boolean Required
      • interval string Required

        The interval is expressed using the crontab syntax

    • incremental object
      Hide incremental attributes Show incremental attributes object
      • enabled boolean Required
      • interval string Required

        The interval is expressed using the crontab syntax

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_scheduling
curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_scheduling' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"scheduling\": {\n        \"access_control\": {\n            \"enabled\": true,\n            \"interval\": \"0 10 0 * * ?\"\n        },\n        \"full\": {\n            \"enabled\": true,\n            \"interval\": \"0 20 0 * * ?\"\n        },\n        \"incremental\": {\n            \"enabled\": false,\n            \"interval\": \"0 30 0 * * ?\"\n        }\n    }\n}"'
{
    "scheduling": {
        "access_control": {
            "enabled": true,
            "interval": "0 10 0 * * ?"
        },
        "full": {
            "enabled": true,
            "interval": "0 20 0 * * ?"
        },
        "incremental": {
            "enabled": false,
            "interval": "0 30 0 * * ?"
        }
    }
}
{
    "scheduling": {
        "full": {
            "enabled": true,
            "interval": "0 10 0 * * ?"
        }
    }
}
Response examples (200)
{
  "result": "updated"
}

Update the connector service type Beta; Added in 8.12.0

PUT /_connector/{connector_id}/_service_type

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

application/json

Body Required

  • service_type string Required

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_service_type
curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_service_type' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"service_type\": \"sharepoint_online\"\n}"'
Request example
{
    "service_type": "sharepoint_online"
}
Response examples (200)
{
  "result": "updated"
}

Update the connector status Technical preview; Added in 8.12.0

PUT /_connector/{connector_id}/_status

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

application/json

Body Required

  • status string Required

    Values are created, needs_configuration, configured, connected, or error.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_status
curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_status' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"status\": \"needs_configuration\"\n}"'
Request example
{
    "status": "needs_configuration"
}
Response examples (200)
{
  "result": "updated"
}









Delete auto-follow patterns Generally available; Added in 6.5.0

DELETE /_ccr/auto_follow/{name}

Delete a collection of cross-cluster replication auto-follow patterns.

Required authorization

  • Cluster privileges: manage_ccr
External documentation

Path parameters

  • name string Required

    The auto-follow pattern collection to delete.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

DELETE /_ccr/auto_follow/{name}
curl \
 --request DELETE 'http://api.example.com/_ccr/auto_follow/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `DELETE /_ccr/auto_follow/my_auto_follow_pattern`, which deletes an auto-follow pattern.
{
  "acknowledged" : true
}




Get follower information Generally available; Added in 6.7.0

GET /{index}/_ccr/info

Get information about all cross-cluster replication follower indices. For example, the results include follower index names, leader index names, replication options, and whether the follower indices are active or paused.

Required authorization

  • Cluster privileges: monitor
External documentation

Path parameters

  • index string | array[string] Required

    A comma-delimited list of follower index patterns.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • follower_indices array[object] Required
      Hide follower_indices attributes Show follower_indices attributes object
      • follower_index string Required
      • leader_index string Required
      • parameters object
        Hide parameters attributes Show parameters attributes object
        • max_outstanding_read_requests number

          The maximum number of outstanding reads requests from the remote cluster.

        • max_outstanding_write_requests number

          The maximum number of outstanding write requests on the follower.

        • max_read_request_operation_count number

          The maximum number of operations to pull per read from the remote cluster.

        • max_read_request_size number | string

        • max_retry_delay string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • max_write_buffer_count number

          The maximum number of operations that can be queued for writing. When this limit is reached, reads from the remote cluster will be deferred until the number of queued operations goes below the limit.

        • max_write_buffer_size number | string

        • max_write_request_operation_count number

          The maximum number of operations per bulk write request executed on the follower.

        • max_write_request_size number | string

        • read_poll_timeout string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • remote_cluster string Required
      • status string Required

        Values are active or paused.

GET /{index}/_ccr/info
curl \
 --request GET 'http://api.example.com/{index}/_ccr/info' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /follower_index/_ccr/info` when the follower index is active.
{
  "follower_indices": [
    {
      "follower_index": "follower_index",
      "remote_cluster": "remote_cluster",
      "leader_index": "leader_index",
      "status": "active",
      "parameters": {
        "max_read_request_operation_count": 5120,
        "max_read_request_size": "32mb",
        "max_outstanding_read_requests": 12,
        "max_write_request_operation_count": 5120,
        "max_write_request_size": "9223372036854775807b",
        "max_outstanding_write_requests": 9,
        "max_write_buffer_count": 2147483647,
        "max_write_buffer_size": "512mb",
        "max_retry_delay": "500ms",
        "read_poll_timeout": "1m"
      }
    }
  ]
}
A successful response from `GET /follower_index/_ccr/info` when the follower index is paused.
{
  "follower_indices": [
    {
      "follower_index": "follower_index",
      "remote_cluster": "remote_cluster",
      "leader_index": "leader_index",
      "status": "paused"
    }
  ]
}




Forget a follower Generally available; Added in 6.7.0

POST /{index}/_ccr/forget_follower

Remove the cross-cluster replication follower retention leases from the leader.

A following index takes out retention leases on its leader index. These leases are used to increase the likelihood that the shards of the leader index retain the history of operations that the shards of the following index need to run replication. When a follower index is converted to a regular index by the unfollow API (either by directly calling the API or by index lifecycle management tasks), these leases are removed. However, removal of the leases can fail, for example when the remote cluster containing the leader index is unavailable. While the leases will eventually expire on their own, their extended existence can cause the leader index to hold more history than necessary and prevent index lifecycle management from performing some operations on the leader index. This API exists to enable manually removing the leases when the unfollow API is unable to do so.

NOTE: This API does not stop replication by a following index. If you use this API with a follower index that is still actively following, the following index will add back retention leases on the leader. The only purpose of this API is to handle the case of failure to remove the following retention leases after the unfollow API is invoked.

External documentation

Path parameters

  • index string Required

    the name of the leader index for which specified follower retention leases should be removed

Query parameters

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

application/json

Body Required

  • follower_cluster string
  • follower_index string
  • follower_index_uuid string
  • leader_remote_cluster string

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • _shards object Required
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number Required
        • status string
      • skipped number
POST /{index}/_ccr/forget_follower
curl \
 --request POST 'http://api.example.com/{index}/_ccr/forget_follower' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"follower_cluster\" : \"\u003cfollower_cluster\u003e\",\n  \"follower_index\" : \"\u003cfollower_index\u003e\",\n  \"follower_index_uuid\" : \"\u003cfollower_index_uuid\u003e\",\n  \"leader_remote_cluster\" : \"\u003cleader_remote_cluster\u003e\"\n}"'
Request example
Run `POST /<leader_index>/_ccr/forget_follower`.
{
  "follower_cluster" : "<follower_cluster>",
  "follower_index" : "<follower_index>",
  "follower_index_uuid" : "<follower_index_uuid>",
  "leader_remote_cluster" : "<leader_remote_cluster>"
}
Response examples (200)
A successful response for removing the follower retention leases from the leader index.
{
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0,
    "failures" : [ ]
  }
}

Get auto-follow patterns Generally available; Added in 6.5.0

GET /_ccr/auto_follow

Get cross-cluster replication auto-follow patterns.

Required authorization

  • Cluster privileges: manage_ccr
External documentation

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • patterns array[object] Required
      Hide patterns attributes Show patterns attributes object
      • name string Required
      • pattern object Required
        Hide pattern attributes Show pattern attributes object
        • active boolean Required
        • remote_cluster string Required

          The remote cluster containing the leader indices to match against.

        • follow_index_pattern string
        • leader_index_patterns array[string] Required
        • leader_index_exclusion_patterns array[string] Required
        • max_outstanding_read_requests number Required

          The maximum number of outstanding reads requests from the remote cluster.

GET /_ccr/auto_follow
curl \
 --request GET 'http://api.example.com/_ccr/auto_follow' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_ccr/auto_follow/my_auto_follow_pattern`, which gets auto-follow patterns.
{
  "patterns": [
    {
      "name": "my_auto_follow_pattern",
      "pattern": {
        "active": true,
        "remote_cluster" : "remote_cluster",
        "leader_index_patterns" :
        [
          "leader_index*"
        ],
        "leader_index_exclusion_patterns":
        [
          "leader_index_001"
        ],
        "follow_index_pattern" : "{{leader_index}}-follower"
      }
    }
  ]
}

Pause an auto-follow pattern Generally available; Added in 7.5.0

POST /_ccr/auto_follow/{name}/pause

Pause a cross-cluster replication auto-follow pattern. When the API returns, the auto-follow pattern is inactive. New indices that are created on the remote cluster and match the auto-follow patterns are ignored.

You can resume auto-following with the resume auto-follow pattern API. When it resumes, the auto-follow pattern is active again and automatically configures follower indices for newly created indices on the remote cluster that match its patterns. Remote indices that were created while the pattern was paused will also be followed, unless they have been deleted or closed in the interim.

Required authorization

  • Cluster privileges: manage_ccr
External documentation

Path parameters

  • name string Required

    The name of the auto-follow pattern to pause.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /_ccr/auto_follow/{name}/pause
curl \
 --request POST 'http://api.example.com/_ccr/auto_follow/{name}/pause' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `POST /_ccr/auto_follow/my_auto_follow_pattern/pause`, which pauses an auto-follow pattern.
{
  "acknowledged" : true
}








Resume a follower Generally available; Added in 6.5.0

POST /{index}/_ccr/resume_follow

Resume a cross-cluster replication follower index that was paused. The follower index could have been paused with the pause follower API. Alternatively it could be paused due to replication that cannot be retried due to failures during following tasks. When this API returns, the follower index will resume fetching operations from the leader index.

External documentation

Path parameters

  • index string Required

    The name of the follow index to resume following.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

application/json

Body

  • max_outstanding_read_requests number
  • max_outstanding_write_requests number
  • max_read_request_operation_count number
  • max_read_request_size string
  • max_retry_delay string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • max_write_buffer_count number
  • max_write_buffer_size string
  • max_write_request_operation_count number
  • max_write_request_size string
  • read_poll_timeout string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /{index}/_ccr/resume_follow
curl \
 --request POST 'http://api.example.com/{index}/_ccr/resume_follow' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"max_read_request_operation_count\" : 1024,\n  \"max_outstanding_read_requests\" : 16,\n  \"max_read_request_size\" : \"1024k\",\n  \"max_write_request_operation_count\" : 32768,\n  \"max_write_request_size\" : \"16k\",\n  \"max_outstanding_write_requests\" : 8,\n  \"max_write_buffer_count\" : 512,\n  \"max_write_buffer_size\" : \"512k\",\n  \"max_retry_delay\" : \"10s\",\n  \"read_poll_timeout\" : \"30s\"\n}"'
Request example
Run `POST /follower_index/_ccr/resume_follow` to resume the follower index.
{
  "max_read_request_operation_count" : 1024,
  "max_outstanding_read_requests" : 16,
  "max_read_request_size" : "1024k",
  "max_write_request_operation_count" : 32768,
  "max_write_request_size" : "16k",
  "max_outstanding_write_requests" : 8,
  "max_write_buffer_count" : 512,
  "max_write_buffer_size" : "512k",
  "max_retry_delay" : "10s",
  "read_poll_timeout" : "30s"
}
Response examples (200)
A successful response from resuming a folower index.
{
  "acknowledged" : true
}

Get cross-cluster replication stats Generally available; Added in 6.5.0

GET /_ccr/stats

This API returns stats about auto-following and the same shard-level stats as the get follower stats API.

Required authorization

  • Cluster privileges: monitor

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

    Values are -1 or 0.

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • auto_follow_stats object Required
      Hide auto_follow_stats attributes Show auto_follow_stats attributes object
      • auto_followed_clusters array[object] Required
        Hide auto_followed_clusters attributes Show auto_followed_clusters attributes object
        • cluster_name string Required
        • last_seen_metadata_version number Required
        • time_since_last_check_millis number

          Time unit for milliseconds

      • number_of_failed_follow_indices number Required

        The number of indices that the auto-follow coordinator failed to automatically follow. The causes of recent failures are captured in the logs of the elected master node and in the auto_follow_stats.recent_auto_follow_errors field.

      • number_of_failed_remote_cluster_state_requests number Required

        The number of times that the auto-follow coordinator failed to retrieve the cluster state from a remote cluster registered in a collection of auto-follow patterns.

      • number_of_successful_follow_indices number Required

        The number of indices that the auto-follow coordinator successfully followed.

      • recent_auto_follow_errors array[object] Required

        An array of objects representing failures by the auto-follow coordinator.

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide recent_auto_follow_errors attributes Show recent_auto_follow_errors attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

    • follow_stats object Required
      Hide follow_stats attribute Show follow_stats attribute object
      • indices array[object] Required
        Hide indices attributes Show indices attributes object
        • index string Required
        • shards array[object] Required

          An array of shard-level following task statistics.

          Hide shards attributes Show shards attributes object
          • bytes_read number Required

            The total of transferred bytes read from the leader. This is only an estimate and does not account for compression if enabled.

          • failed_read_requests number Required

            The number of failed reads.

          • failed_write_requests number Required

            The number of failed bulk write requests on the follower.

          • fatal_exception object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • follower_aliases_version number Required
          • follower_global_checkpoint number Required

            The current global checkpoint on the follower. The difference between the leader_global_checkpoint and the follower_global_checkpoint is an indication of how much the follower is lagging the leader.

          • follower_index string Required

            The name of the follower index.

          • follower_mapping_version number Required
          • follower_max_seq_no number Required
          • follower_settings_version number Required
          • last_requested_seq_no number Required
          • leader_global_checkpoint number Required

            The current global checkpoint on the leader known to the follower task.

          • leader_index string Required

            The name of the index in the leader cluster being followed.

          • leader_max_seq_no number Required
          • operations_read number Required

            The total number of operations read from the leader.

          • operations_written number Required

            The number of operations written on the follower.

          • outstanding_read_requests number Required

            The number of active read requests from the follower.

          • outstanding_write_requests number Required

            The number of active bulk write requests on the follower.

          • read_exceptions array[object] Required

            An array of objects representing failed reads.

          • remote_cluster string Required

            The remote cluster containing the leader index.

          • shard_id number Required

            The numerical shard ID, with values from 0 to one less than the number of replicas.

          • successful_read_requests number Required

            The number of successful fetches.

          • successful_write_requests number Required

            The number of bulk write requests run on the follower.

          • time_since_last_read string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • time_since_last_read_millis
          • total_read_remote_exec_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_read_remote_exec_time_millis
          • total_read_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_read_time_millis
          • total_write_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_write_time_millis
          • write_buffer_operation_count number Required

            The number of write operations queued on the follower.

          • write_buffer_size_in_bytes
GET /_ccr/stats
curl \
 --request GET 'http://api.example.com/_ccr/stats' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_ccr/stats` that returns cross-cluster replication stats.
{
  "auto_follow_stats" : {
    "number_of_failed_follow_indices" : 0,
    "number_of_failed_remote_cluster_state_requests" : 0,
    "number_of_successful_follow_indices" : 1,
    "recent_auto_follow_errors" : [],
    "auto_followed_clusters" : []
  },
  "follow_stats" : {
    "indices" : [
      {
        "index" : "follower_index",
        "total_global_checkpoint_lag" : 256,
        "shards" : [
          {
            "remote_cluster" : "remote_cluster",
            "leader_index" : "leader_index",
            "follower_index" : "follower_index",
            "shard_id" : 0,
            "leader_global_checkpoint" : 1024,
            "leader_max_seq_no" : 1536,
            "follower_global_checkpoint" : 768,
            "follower_max_seq_no" : 896,
            "last_requested_seq_no" : 897,
            "outstanding_read_requests" : 8,
            "outstanding_write_requests" : 2,
            "write_buffer_operation_count" : 64,
            "follower_mapping_version" : 4,
            "follower_settings_version" : 2,
            "follower_aliases_version" : 8,
            "total_read_time_millis" : 32768,
            "total_read_remote_exec_time_millis" : 16384,
            "successful_read_requests" : 32,
            "failed_read_requests" : 0,
            "operations_read" : 896,
            "bytes_read" : 32768,
            "total_write_time_millis" : 16384,
            "write_buffer_size_in_bytes" : 1536,
            "successful_write_requests" : 16,
            "failed_write_requests" : 0,
            "operations_written" : 832,
            "read_exceptions" : [ ],
            "time_since_last_read_millis" : 8
          }
        ]
      }
    ]
  }
}

Unfollow an index Generally available; Added in 6.5.0

POST /{index}/_ccr/unfollow

Convert a cross-cluster replication follower index to a regular index. The API stops the following task associated with a follower index and removes index metadata and settings associated with cross-cluster replication. The follower index must be paused and closed before you call the unfollow API.


Currently cross-cluster replication does not support converting an existing regular index to a follower index. Converting a follower index to a regular index is an irreversible operation.

Required authorization

  • Index privileges: manage_follow_index
External documentation

Path parameters

  • index string Required

    The name of the follower index.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /{index}/_ccr/unfollow
curl \
 --request POST 'http://api.example.com/{index}/_ccr/unfollow' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `POST /follower_index/_ccr/unfollow`.
{
  "acknowledged" : true
}

Get data streams Generally available; Added in 7.9.0

GET /_data_stream/{name}

Get information about one or more data streams.

Required authorization

  • Index privileges: view_index_metadata

Path parameters

  • name string | array[string] Required

    Comma-separated list of data stream names used to limit the request. Wildcard (*) expressions are supported. If omitted, all data streams are returned.

Query parameters

  • expand_wildcards string | array[string]

    Type of data stream that wildcard patterns can match. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • include_defaults boolean

    If true, returns all relevant default configurations for the index template.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • verbose boolean

    Whether the maximum timestamp for each data stream should be calculated and returned.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • data_streams array[object] Required
      Hide data_streams attributes Show data_streams attributes object
      • _meta object
        Hide _meta attribute Show _meta attribute object
        • * object Additional properties
      • allow_custom_routing boolean

        If true, the data stream allows custom routing on write request.

      • failure_store object
        Hide failure_store attributes Show failure_store attributes object
        • enabled boolean Required
        • indices array[object] Required
          Hide indices attributes Show indices attributes object
          • index_name string Required
          • index_uuid string Required
          • ilm_policy string
          • managed_by string

            Values are Index Lifecycle Management, Data stream lifecycle, or Unmanaged.

          • prefer_ilm boolean

            Indicates if ILM should take precedence over DSL in case both are configured to manage this index.

        • rollover_on_write boolean Required
      • generation number Required

        Current generation for the data stream. This number acts as a cumulative count of the stream’s rollovers, starting at 1.

      • hidden boolean Required

        If true, the data stream is hidden.

      • ilm_policy string
      • next_generation_managed_by string Required

        Values are Index Lifecycle Management, Data stream lifecycle, or Unmanaged.

      • prefer_ilm boolean Required

        Indicates if ILM should take precedence over DSL in case both are configured to managed this data stream.

      • indices array[object] Required

        Array of objects containing information about the data stream’s backing indices. The last item in this array contains information about the stream’s current write index.

        Hide indices attributes Show indices attributes object
        • index_name string Required
        • index_uuid string Required
        • ilm_policy string
        • managed_by string

          Values are Index Lifecycle Management, Data stream lifecycle, or Unmanaged.

        • prefer_ilm boolean

          Indicates if ILM should take precedence over DSL in case both are configured to manage this index.

      • lifecycle object

        Data stream lifecycle with rollover can be used to display the configuration including the default rollover conditions, if asked.

        Hide lifecycle attributes Show lifecycle attributes object
        • data_retention string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • downsampling object
          Hide downsampling attribute Show downsampling attribute object
          • rounds array[object] Required

            The list of downsampling rounds to execute as part of this downsampling configuration

        • enabled boolean

          If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

        • rollover object
          Hide rollover attributes Show rollover attributes object
          • min_age string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • max_age string
          • min_docs number
          • max_docs number
          • min_size
          • max_size
          • min_primary_shard_size
          • max_primary_shard_size
          • min_primary_shard_docs number
          • max_primary_shard_docs number
      • name string Required
      • replicated boolean

        If true, the data stream is created and managed by cross-cluster replication and the local cluster can not write into this data stream or change its mappings.

      • rollover_on_write boolean Required

        If true, the next write to this data stream will trigger a rollover first and the document will be indexed in the new backing index. If the rollover fails the indexing request will fail too.

      • status string Required

        Values are green, GREEN, yellow, YELLOW, red, or RED.

      • system boolean Generally available; Added in 7.10.0

        If true, the data stream is created and managed by an Elastic stack component and cannot be modified through normal user interaction.

      • template string Required
      • timestamp_field object Required
        Hide timestamp_field attribute Show timestamp_field attribute object
        • name string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

GET /_data_stream/{name}
curl \
 --request GET 'http://api.example.com/_data_stream/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response for retrieving information about a data stream.
{
  "data_streams": [
    {
      "name": "my-data-stream",
      "timestamp_field": {
        "name": "@timestamp"
      },
      "indices": [
        {
          "index_name": ".ds-my-data-stream-2099.03.07-000001",
          "index_uuid": "xCEhwsp8Tey0-FLNFYVwSg",
          "prefer_ilm": true,
          "ilm_policy": "my-lifecycle-policy",
          "managed_by": "Index Lifecycle Management"
        },
        {
          "index_name": ".ds-my-data-stream-2099.03.08-000002",
          "index_uuid": "PA_JquKGSiKcAKBA8DJ5gw",
          "prefer_ilm": true,
          "ilm_policy": "my-lifecycle-policy",
          "managed_by": "Index Lifecycle Management"
        }
      ],
      "generation": 2,
      "_meta": {
        "my-meta-field": "foo"
      },
      "status": "GREEN",
      "next_generation_managed_by": "Index Lifecycle Management",
      "prefer_ilm": true,
      "template": "my-index-template",
      "ilm_policy": "my-lifecycle-policy",
      "hidden": false,
      "system": false,
      "allow_custom_routing": false,
      "replicated": false,
      "rollover_on_write": false
    },
    {
      "name": "my-data-stream-two",
      "timestamp_field": {
        "name": "@timestamp"
      },
      "indices": [
        {
          "index_name": ".ds-my-data-stream-two-2099.03.08-000001",
          "index_uuid": "3liBu2SYS5axasRt6fUIpA",
          "prefer_ilm": true,
          "ilm_policy": "my-lifecycle-policy",
          "managed_by": "Index Lifecycle Management"
        }
      ],
      "generation": 1,
      "_meta": {
        "my-meta-field": "foo"
      },
      "status": "YELLOW",
      "next_generation_managed_by": "Index Lifecycle Management",
      "prefer_ilm": true,
      "template": "my-index-template",
      "ilm_policy": "my-lifecycle-policy",
      "hidden": false,
      "system": false,
      "allow_custom_routing": false,
      "replicated": false,
      "rollover_on_write": false
    }
  ]
}
















Get data stream lifecycles Generally available; Added in 8.11.0

GET /_data_stream/{name}/_lifecycle

Get the data stream lifecycle configuration of one or more data streams.

External documentation

Path parameters

  • name string | array[string] Required

    Comma-separated list of data streams to limit the request. Supports wildcards (*). To target all data streams, omit this parameter or use * or _all.

Query parameters

  • expand_wildcards string | array[string]

    Type of data stream that wildcard patterns can match. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • include_defaults boolean

    If true, return all default settings in the response.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • data_streams array[object] Required
      Hide data_streams attributes Show data_streams attributes object
      • name string Required
      • lifecycle object

        Data stream lifecycle with rollover can be used to display the configuration including the default rollover conditions, if asked.

        Hide lifecycle attributes Show lifecycle attributes object
        • data_retention string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • downsampling object
          Hide downsampling attribute Show downsampling attribute object
          • rounds array[object] Required

            The list of downsampling rounds to execute as part of this downsampling configuration

        • enabled boolean

          If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

        • rollover object
          Hide rollover attributes Show rollover attributes object
          • min_age string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • max_age string
          • min_docs number
          • max_docs number
          • min_size
          • max_size
          • min_primary_shard_size
          • max_primary_shard_size
          • min_primary_shard_docs number
          • max_primary_shard_docs number
GET /_data_stream/{name}/_lifecycle
curl \
 --request GET 'http://api.example.com/_data_stream/{name}/_lifecycle' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_data_stream/{name}/_lifecycle?human&pretty`.
{
  "data_streams": [
    {
      "name": "my-data-stream-1",
      "lifecycle": {
        "enabled": true,
        "data_retention": "7d"
      }
    },
    {
      "name": "my-data-stream-2",
      "lifecycle": {
        "enabled": true,
        "data_retention": "7d"
      }
    }
  ]
}

Update data stream lifecycles Generally available; Added in 8.11.0

PUT /_data_stream/{name}/_lifecycle

Update the data stream lifecycle of the specified data streams.

External documentation

Path parameters

  • name string | array[string] Required

    Comma-separated list of data streams used to limit the request. Supports wildcards (*). To target all data streams use * or _all.

Query parameters

  • expand_wildcards string | array[string]

    Type of data stream that wildcard patterns can match. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

application/json

Body

  • data_retention string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • downsampling object
    Hide downsampling attribute Show downsampling attribute object
    • rounds array[object] Required

      The list of downsampling rounds to execute as part of this downsampling configuration

      Hide rounds attributes Show rounds attributes object
      • after string Required

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • config object Required
        Hide config attribute Show config attribute object
        • fixed_interval string Required

          A date histogram interval. Similar to Duration with additional units: w (week), M (month), q (quarter) and y (year)

  • enabled boolean

    If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_data_stream/{name}/_lifecycle
curl \
 --request PUT 'http://api.example.com/_data_stream/{name}/_lifecycle' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"data_retention\": \"7d\"\n}"'
{
  "data_retention": "7d"
}
This example configures two downsampling rounds.
{
    "downsampling": [
      {
        "after": "1d",
        "fixed_interval": "10m"
      },
      {
        "after": "7d",
        "fixed_interval": "1d"
      }
    ]
}
Response examples (200)
A successful response for configuring a data stream lifecycle.
{
  "acknowledged": true
}




Get the status for a data stream lifecycle Generally available; Added in 8.11.0

GET /{index}/_lifecycle/explain

Get information about an index or data stream's current data stream lifecycle status, such as time since index creation, time since rollover, the lifecycle configuration managing the index, or any errors encountered during lifecycle execution.

External documentation

Path parameters

  • index string | array[string] Required

    The name of the index to explain

Query parameters

  • include_defaults boolean

    indicates if the API should return the default values the system uses for the index's lifecycle

  • master_timeout string

    Specify timeout for connection to master

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • indices object Required
      Hide indices attribute Show indices attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • index string Required
        • managed_by_lifecycle boolean Required
        • index_creation_date_millis number

          Time unit for milliseconds

        • time_since_index_creation string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • rollover_date_millis number

          Time unit for milliseconds

        • time_since_rollover string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • lifecycle object

          Data stream lifecycle with rollover can be used to display the configuration including the default rollover conditions, if asked.

          Hide lifecycle attributes Show lifecycle attributes object
          • data_retention string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • downsampling object
            Hide downsampling attribute Show downsampling attribute object
            • rounds array[object] Required

              The list of downsampling rounds to execute as part of this downsampling configuration

          • enabled boolean

            If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

          • rollover object
            Hide rollover attributes Show rollover attributes object
            • min_age string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • max_age string
            • min_docs number
            • max_docs number
            • min_size
            • max_size
            • min_primary_shard_size
            • max_primary_shard_size
            • min_primary_shard_docs number
            • max_primary_shard_docs number
        • generation_time string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • error string
GET /{index}/_lifecycle/explain
curl \
 --request GET 'http://api.example.com/{index}/_lifecycle/explain' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET .ds-metrics-2023.03.22-000001/_lifecycle/explain`, which retrieves the lifecycle status for a data stream backing index. If the index is managed by a data stream lifecycle, the API will show the `managed_by_lifecycle` field set to `true` and the rest of the response will contain information about the lifecycle execution status for this index.
{
  "indices": {
    ".ds-metrics-2023.03.22-000001": {
      "index" : ".ds-metrics-2023.03.22-000001",
      "managed_by_lifecycle" : true,
      "index_creation_date_millis" : 1679475563571,
      "time_since_index_creation" : "843ms",
      "rollover_date_millis" : 1679475564293,
      "time_since_rollover" : "121ms",
      "lifecycle" : { },
      "generation_time" : "121ms"
  }
}
The API reports any errors related to the lifecycle execution for the target index.
{
  "indices": {
    ".ds-metrics-2023.03.22-000001": {
      "index" : ".ds-metrics-2023.03.22-000001",
      "managed_by_lifecycle" : true,
      "index_creation_date_millis" : 1679475563571,
      "time_since_index_creation" : "843ms",
      "lifecycle" : {
        "enabled": true
      },
      "error": "{\"type\":\"validation_exception\",\"reason\":\"Validation Failed: 1: this action would add [2] shards, but this cluster
currently has [4]/[3] maximum normal shards open;\"}"
  }
}

Get data stream lifecycle stats Generally available; Added in 8.12.0

GET /_lifecycle/stats

Get statistics about the data streams that are managed by a data stream lifecycle.

Required authorization

  • Cluster privileges: monitor
External documentation

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • data_stream_count number Required

      The count of data streams currently being managed by the data stream lifecycle.

    • data_streams array[object] Required

      Information about the data streams that are managed by the data stream lifecycle.

      Hide data_streams attributes Show data_streams attributes object
      • backing_indices_in_error number Required

        The count of the backing indices for the data stream.

      • backing_indices_in_total number Required

        The count of the backing indices for the data stream that have encountered an error.

      • name string Required
    • last_run_duration_in_millis number

      Time unit for milliseconds

    • time_between_starts_in_millis number

      Time unit for milliseconds

GET /_lifecycle/stats
curl \
 --request GET 'http://api.example.com/_lifecycle/stats' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response for `GET _lifecycle/stats?human&pretty`
{
  "last_run_duration_in_millis": 2,
  "last_run_duration": "2ms",
  "time_between_starts_in_millis": 9998,
  "time_between_starts": "9.99s",
  "data_streams_count": 2,
  "data_streams": [
    {
      "name": "my-data-stream",
      "backing_indices_in_total": 2,
      "backing_indices_in_error": 0
    },
    {
      "name": "my-other-stream",
      "backing_indices_in_total": 2,
      "backing_indices_in_error": 1
    }
  ]
}

Get data streams Generally available; Added in 7.9.0

GET /_data_stream

Get information about one or more data streams.

Required authorization

  • Index privileges: view_index_metadata

Query parameters

  • expand_wildcards string | array[string]

    Type of data stream that wildcard patterns can match. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • include_defaults boolean

    If true, returns all relevant default configurations for the index template.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • verbose boolean

    Whether the maximum timestamp for each data stream should be calculated and returned.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • data_streams array[object] Required
      Hide data_streams attributes Show data_streams attributes object
      • _meta object
        Hide _meta attribute Show _meta attribute object
        • * object Additional properties
      • allow_custom_routing boolean

        If true, the data stream allows custom routing on write request.

      • failure_store object
        Hide failure_store attributes Show failure_store attributes object
        • enabled boolean Required
        • indices array[object] Required
          Hide indices attributes Show indices attributes object
          • index_name string Required
          • index_uuid string Required
          • ilm_policy string
          • managed_by string

            Values are Index Lifecycle Management, Data stream lifecycle, or Unmanaged.

          • prefer_ilm boolean

            Indicates if ILM should take precedence over DSL in case both are configured to manage this index.

        • rollover_on_write boolean Required
      • generation number Required

        Current generation for the data stream. This number acts as a cumulative count of the stream’s rollovers, starting at 1.

      • hidden boolean Required

        If true, the data stream is hidden.

      • ilm_policy string
      • next_generation_managed_by string Required

        Values are Index Lifecycle Management, Data stream lifecycle, or Unmanaged.

      • prefer_ilm boolean Required

        Indicates if ILM should take precedence over DSL in case both are configured to managed this data stream.

      • indices array[object] Required

        Array of objects containing information about the data stream’s backing indices. The last item in this array contains information about the stream’s current write index.

        Hide indices attributes Show indices attributes object
        • index_name string Required
        • index_uuid string Required
        • ilm_policy string
        • managed_by string

          Values are Index Lifecycle Management, Data stream lifecycle, or Unmanaged.

        • prefer_ilm boolean

          Indicates if ILM should take precedence over DSL in case both are configured to manage this index.

      • lifecycle object

        Data stream lifecycle with rollover can be used to display the configuration including the default rollover conditions, if asked.

        Hide lifecycle attributes Show lifecycle attributes object
        • data_retention string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • downsampling object
          Hide downsampling attribute Show downsampling attribute object
          • rounds array[object] Required

            The list of downsampling rounds to execute as part of this downsampling configuration

        • enabled boolean

          If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

        • rollover object
          Hide rollover attributes Show rollover attributes object
          • min_age string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • max_age string
          • min_docs number
          • max_docs number
          • min_size
          • max_size
          • min_primary_shard_size
          • max_primary_shard_size
          • min_primary_shard_docs number
          • max_primary_shard_docs number
      • name string Required
      • replicated boolean

        If true, the data stream is created and managed by cross-cluster replication and the local cluster can not write into this data stream or change its mappings.

      • rollover_on_write boolean Required

        If true, the next write to this data stream will trigger a rollover first and the document will be indexed in the new backing index. If the rollover fails the indexing request will fail too.

      • status string Required

        Values are green, GREEN, yellow, YELLOW, red, or RED.

      • system boolean Generally available; Added in 7.10.0

        If true, the data stream is created and managed by an Elastic stack component and cannot be modified through normal user interaction.

      • template string Required
      • timestamp_field object Required
        Hide timestamp_field attribute Show timestamp_field attribute object
        • name string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

GET /_data_stream
curl \
 --request GET 'http://api.example.com/_data_stream' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response for retrieving information about a data stream.
{
  "data_streams": [
    {
      "name": "my-data-stream",
      "timestamp_field": {
        "name": "@timestamp"
      },
      "indices": [
        {
          "index_name": ".ds-my-data-stream-2099.03.07-000001",
          "index_uuid": "xCEhwsp8Tey0-FLNFYVwSg",
          "prefer_ilm": true,
          "ilm_policy": "my-lifecycle-policy",
          "managed_by": "Index Lifecycle Management"
        },
        {
          "index_name": ".ds-my-data-stream-2099.03.08-000002",
          "index_uuid": "PA_JquKGSiKcAKBA8DJ5gw",
          "prefer_ilm": true,
          "ilm_policy": "my-lifecycle-policy",
          "managed_by": "Index Lifecycle Management"
        }
      ],
      "generation": 2,
      "_meta": {
        "my-meta-field": "foo"
      },
      "status": "GREEN",
      "next_generation_managed_by": "Index Lifecycle Management",
      "prefer_ilm": true,
      "template": "my-index-template",
      "ilm_policy": "my-lifecycle-policy",
      "hidden": false,
      "system": false,
      "allow_custom_routing": false,
      "replicated": false,
      "rollover_on_write": false
    },
    {
      "name": "my-data-stream-two",
      "timestamp_field": {
        "name": "@timestamp"
      },
      "indices": [
        {
          "index_name": ".ds-my-data-stream-two-2099.03.08-000001",
          "index_uuid": "3liBu2SYS5axasRt6fUIpA",
          "prefer_ilm": true,
          "ilm_policy": "my-lifecycle-policy",
          "managed_by": "Index Lifecycle Management"
        }
      ],
      "generation": 1,
      "_meta": {
        "my-meta-field": "foo"
      },
      "status": "YELLOW",
      "next_generation_managed_by": "Index Lifecycle Management",
      "prefer_ilm": true,
      "template": "my-index-template",
      "ilm_policy": "my-lifecycle-policy",
      "hidden": false,
      "system": false,
      "allow_custom_routing": false,
      "replicated": false,
      "rollover_on_write": false
    }
  ]
}

Convert an index alias to a data stream Generally available; Added in 7.9.0

POST /_data_stream/_migrate/{name}

Converts an index alias to a data stream. You must have a matching index template that is data stream enabled. The alias must meet the following criteria: The alias must have a write index; All indices for the alias must have a @timestamp field mapping of a date or date_nanos field type; The alias must not have any filters; The alias must not use custom routing. If successful, the request removes the alias and creates a data stream with the same name. The indices for the alias become hidden backing indices for the stream. The write index for the alias becomes the write index for the stream.

Required authorization

  • Index privileges: manage

Path parameters

  • name string Required

    Name of the index alias to convert to a data stream.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /_data_stream/_migrate/{name}
curl \
 --request POST 'http://api.example.com/_data_stream/_migrate/{name}' \
 --header "Authorization: $API_KEY"

Update data streams Generally available; Added in 7.16.0

POST /_data_stream/_modify

Performs one or more data stream modification actions in a single atomic operation.

application/json

Body Required

  • actions array[object] Required

    Actions to perform.

    Hide actions attributes Show actions attributes object
    • add_backing_index object
      Hide add_backing_index attributes Show add_backing_index attributes object
      • data_stream string Required
      • index string Required
    • remove_backing_index object
      Hide remove_backing_index attributes Show remove_backing_index attributes object
      • data_stream string Required
      • index string Required

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /_data_stream/_modify
curl \
 --request POST 'http://api.example.com/_data_stream/_modify' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"actions\": [\n    {\n      \"remove_backing_index\": {\n        \"data_stream\": \"my-data-stream\",\n        \"index\": \".ds-my-data-stream-2023.07.26-000001\"\n      }\n    },\n    {\n      \"add_backing_index\": {\n        \"data_stream\": \"my-data-stream\",\n        \"index\": \".ds-my-data-stream-2023.07.26-000001-downsample\"\n      }\n    }\n  ]\n}"'
Request example
An example body for a `POST _data_stream/_modify` request.
{
  "actions": [
    {
      "remove_backing_index": {
        "data_stream": "my-data-stream",
        "index": ".ds-my-data-stream-2023.07.26-000001"
      }
    },
    {
      "add_backing_index": {
        "data_stream": "my-data-stream",
        "index": ".ds-my-data-stream-2023.07.26-000001-downsample"
      }
    }
  ]
}













Bulk index or delete documents Generally available

PUT /{index}/_bulk

Perform multiple index, create, delete, and update actions in a single request. This reduces overhead and can greatly increase indexing speed.

If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:

  • To use the create action, you must have the create_doc, create, index, or write index privilege. Data streams support only the create action.
  • To use the index action, you must have the create, index, or write index privilege.
  • To use the delete action, you must have the delete or write index privilege.
  • To use the update action, you must have the index or write index privilege.
  • To automatically create a data stream or index with a bulk API request, you must have the auto_configure, create_index, or manage index privilege.
  • To make the result of a bulk operation visible to search using the refresh parameter, you must have the maintenance or manage index privilege.

Automatic data stream creation requires a matching index template with data stream enabled.

The actions are specified in the request body using a newline delimited JSON (NDJSON) structure:

action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n

The index and create actions expect a source on the next line and have the same semantics as the op_type parameter in the standard index API. A create action fails if a document with the same ID already exists in the target An index action adds or replaces a document as necessary.

NOTE: Data streams support only the create action. To update or delete a document in a data stream, you must target the backing index containing the document.

An update action expects that the partial doc, upsert, and script and its options are specified on the next line.

A delete action does not expect a source on the next line and has the same semantics as the standard delete API.

NOTE: The final line of data must end with a newline character (\n). Each newline character may be preceded by a carriage return (\r). When sending NDJSON data to the _bulk endpoint, use a Content-Type header of application/json or application/x-ndjson. Because this format uses literal newline characters (\n) as delimiters, make sure that the JSON actions and sources are not pretty printed.

If you provide a target in the request path, it is used for any actions that don't explicitly specify an _index argument.

A note on the format: the idea here is to make processing as fast as possible. As some of the actions are redirected to other shards on other nodes, only action_meta_data is parsed on the receiving node side.

Client libraries using this protocol should try and strive to do something similar on the client side, and reduce buffering as much as possible.

There is no "correct" number of actions to perform in a single bulk request. Experiment with different settings to find the optimal size for your particular workload. Note that Elasticsearch limits the maximum size of a HTTP request to 100mb by default so clients must ensure that no request exceeds this size. It is not possible to index a single document that exceeds the size limit, so you must pre-process any such documents into smaller pieces before sending them to Elasticsearch. For instance, split documents into pages or chapters before indexing them, or store raw binary data in a system outside Elasticsearch and replace the raw data with a link to the external system in the documents that you send to Elasticsearch.

Client suppport for bulk requests

Some of the officially supported clients provide helpers to assist with bulk requests and reindexing:

  • Go: Check out esutil.BulkIndexer
  • Perl: Check out Search::Elasticsearch::Client::5_0::Bulk and Search::Elasticsearch::Client::5_0::Scroll
  • Python: Check out elasticsearch.helpers.*
  • JavaScript: Check out client.helpers.*
  • .NET: Check out BulkAllObservable
  • PHP: Check out bulk indexing.

Submitting bulk requests with cURL

If you're providing text file input to curl, you must use the --data-binary flag instead of plain -d. The latter doesn't preserve newlines. For example:

$ cat requests
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
$ curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests"; echo
{"took":7, "errors": false, "items":[{"index":{"_index":"test","_id":"1","_version":1,"result":"created","forced_refresh":false}}]}

Optimistic concurrency control

Each index and delete action within a bulk API call may include the if_seq_no and if_primary_term parameters in their respective action and meta data lines. The if_seq_no and if_primary_term parameters control how operations are run, based on the last modification to existing documents. See Optimistic concurrency control for more details.

Versioning

Each bulk item can include the version value using the version field. It automatically follows the behavior of the index or delete operation based on the _version mapping. It also support the version_type.

Routing

Each bulk item can include the routing value using the routing field. It automatically follows the behavior of the index or delete operation based on the _routing mapping.

NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.

Wait for active shards

When making bulk calls, you can set the wait_for_active_shards parameter to require a minimum number of shard copies to be active before starting to process the bulk request.

Refresh

Control when the changes made by this request are visible to search.

NOTE: Only the shards that receive the bulk request will be affected by refresh. Imagine a _bulk?refresh=wait_for request with three documents in it that happen to be routed to different shards in an index with five shards. The request will only wait for those three shards to refresh. The other two shards that make up the index do not participate in the _bulk request at all.

Path parameters

  • index string Required

    The name of the data stream, index, or index alias to perform bulk actions on.

Query parameters

  • include_source_on_error boolean

    True or false if to include the document source in the error message in case of parsing errors.

  • list_executed_pipelines boolean

    If true, the response will include the ingest pipelines that were run for each index or create.

  • pipeline string

    The pipeline identifier to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter.

  • refresh string

    If true, Elasticsearch refreshes the affected shards to make this operation visible to search. If wait_for, wait for a refresh to make this operation visible to search. If false, do nothing with refreshes. Valid values: true, false, wait_for.

    Values are true, false, or wait_for.

  • routing string

    A custom value that is used to route operations to a specific shard.

  • _source boolean | string | array[string]

    Indicates whether to return the _source field (true or false) or contains a list of fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter. If the _source parameter is false, this parameter is ignored.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter. If the _source parameter is false, this parameter is ignored.

  • timeout string

    The period each action waits for the following operations: automatic index creation, dynamic mapping updates, and waiting for active shards. The default is 1m (one minute), which guarantees Elasticsearch waits for at least the timeout before failing. The actual wait time could be longer, particularly when multiple waits occur.

    Values are -1 or 0.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default is 1, which waits for each primary shard to be active.

    Values are all or index-setting.

  • require_alias boolean

    If true, the request's actions must target an index alias.

  • require_data_stream boolean

    If true, the request's actions must target a data stream (existing or to be created).

application/json

Body object Required

One of:
  • index object
    Hide index attributes Show index attributes object
    • _id string
    • _index string
    • routing string
    • if_primary_term number
    • if_seq_no number
    • version number
    • version_type string

      Values are internal, external, external_gte, or force.

    • dynamic_templates object

      A map from the full name of fields to the name of dynamic templates. It defaults to an empty map. If a name matches a dynamic template, that template will be applied regardless of other match predicates defined in the template. If a field is already defined in the mapping, then this parameter won't be used.

      Hide dynamic_templates attribute Show dynamic_templates attribute object
      • * string Additional properties
    • pipeline string

      The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter.

    • require_alias boolean

      If true, the request's actions must target an index alias.

  • create object
    Hide create attributes Show create attributes object
    • _id string
    • _index string
    • routing string
    • if_primary_term number
    • if_seq_no number
    • version number
    • version_type string

      Values are internal, external, external_gte, or force.

    • dynamic_templates object

      A map from the full name of fields to the name of dynamic templates. It defaults to an empty map. If a name matches a dynamic template, that template will be applied regardless of other match predicates defined in the template. If a field is already defined in the mapping, then this parameter won't be used.

      Hide dynamic_templates attribute Show dynamic_templates attribute object
      • * string Additional properties
    • pipeline string

      The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter.

    • require_alias boolean

      If true, the request's actions must target an index alias.

  • update object
    Hide update attributes Show update attributes object
    • _id string
    • _index string
    • routing string
    • if_primary_term number
    • if_seq_no number
    • version number
    • version_type string

      Values are internal, external, external_gte, or force.

    • require_alias boolean

      If true, the request's actions must target an index alias.

    • retry_on_conflict number

      The number of times an update should be retried in the case of a version conflict.

  • delete object
    Hide delete attributes Show delete attributes object
    • _id string
    • _index string
    • routing string
    • if_primary_term number
    • if_seq_no number
    • version number
    • version_type string

      Values are internal, external, external_gte, or force.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • errors boolean Required

      If true, one or more of the operations in the bulk request did not complete successfully.

    • items array[object] Required

      The result of each operation in the bulk request, in the order they were submitted.

      Hide items attribute Show items attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • _id string | null

          The document ID associated with the operation.

        • _index string Required

          The name of the index associated with the operation. If the operation targeted a data stream, this is the backing index into which the document was written.

        • status number Required

          The HTTP status code returned for the operation.

        • failure_store string

          Values are not_applicable_or_unknown, used, not_enabled, or failed.

        • error object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide error attributes Show error attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • _primary_term number

          The primary term assigned to the document for the operation. This property is returned only for successful operations.

        • result string

          The result of the operation. Successful values are created, deleted, and updated.

        • _seq_no number
        • _shards object
          Hide _shards attributes Show _shards attributes object
          • failed number Required
          • successful number Required
          • total number Required
          • failures array[object]
          • skipped number
        • _version number
        • forced_refresh boolean
        • get object
          Hide get attributes Show get attributes object
          • fields object
            Hide fields attribute Show fields attribute object
            • * object Additional properties
          • found boolean Required
          • _seq_no number
          • _primary_term number
          • _routing string
          • _source object
            Hide _source attribute Show _source attribute object
            • * object Additional properties
    • took number Required

      The length of time, in milliseconds, it took to process the bulk request.

    • ingest_took number
PUT /{index}/_bulk
curl \
 --request PUT 'http://api.example.com/{index}/_bulk' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{ \"index\" : { \"_index\" : \"test\", \"_id\" : \"1\" } }\n{ \"field1\" : \"value1\" }\n{ \"delete\" : { \"_index\" : \"test\", \"_id\" : \"2\" } }\n{ \"create\" : { \"_index\" : \"test\", \"_id\" : \"3\" } }\n{ \"field1\" : \"value3\" }\n{ \"update\" : {\"_id\" : \"1\", \"_index\" : \"test\"} }\n{ \"doc\" : {\"field2\" : \"value2\"} }"'
Run `POST _bulk` to perform multiple operations.
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_id" : "2" } }
{ "create" : { "_index" : "test", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }
When you run `POST _bulk` and use the `update` action, you can use `retry_on_conflict` as a field in the action itself (not in the extra payload line) to specify how many times an update should be retried in the case of a version conflict.
{ "update" : {"_id" : "1", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_index" : "index1", "retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}
To return only information about failed operations, run `POST /_bulk?filter_path=items.*.error`.
{ "update": {"_id": "5", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "update": {"_id": "6", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "create": {"_id": "7", "_index": "index1"} }
{ "my_field": "foo" }
Run `POST /_bulk` to perform a bulk request that consists of index and create actions with the `dynamic_templates` parameter. The bulk request creates two new fields `work_location` and `home_location` with type `geo_point` according to the `dynamic_templates` parameter. However, the `raw_location` field is created using default dynamic mapping rules, as a text field in that case since it is supplied as a string in the JSON document.
{ "index" : { "_index" : "my_index", "_id" : "1", "dynamic_templates": {"work_location": "geo_point"}} }
{ "field" : "value1", "work_location": "41.12,-71.34", "raw_location": "41.12,-71.34"}
{ "create" : { "_index" : "my_index", "_id" : "2", "dynamic_templates": {"home_location": "geo_point"}} }
{ "field" : "value2", "home_location": "41.12,-71.34"}
Response examples (200)
{
   "took": 30,
   "errors": false,
   "items": [
      {
         "index": {
            "_index": "test",
            "_id": "1",
            "_version": 1,
            "result": "created",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "status": 201,
            "_seq_no" : 0,
            "_primary_term": 1
         }
      },
      {
         "delete": {
            "_index": "test",
            "_id": "2",
            "_version": 1,
            "result": "not_found",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "status": 404,
            "_seq_no" : 1,
            "_primary_term" : 2
         }
      },
      {
         "create": {
            "_index": "test",
            "_id": "3",
            "_version": 1,
            "result": "created",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "status": 201,
            "_seq_no" : 2,
            "_primary_term" : 3
         }
      },
      {
         "update": {
            "_index": "test",
            "_id": "1",
            "_version": 2,
            "result": "updated",
            "_shards": {
                "total": 2,
                "successful": 1,
                "failed": 0
            },
            "status": 200,
            "_seq_no" : 3,
            "_primary_term" : 4
         }
      }
   ]
}
If you run `POST /_bulk` with operations that update non-existent documents, the operations cannot complete successfully. The API returns a response with an `errors` property value `true`. The response also includes an error object for any failed operations. The error object contains additional information about the failure, such as the error type and reason.
{
  "took": 486,
  "errors": true,
  "items": [
    {
      "update": {
        "_index": "index1",
        "_id": "5",
        "status": 404,
        "error": {
          "type": "document_missing_exception",
          "reason": "[5]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    },
    {
      "update": {
        "_index": "index1",
        "_id": "6",
        "status": 404,
        "error": {
          "type": "document_missing_exception",
          "reason": "[6]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    },
    {
      "create": {
        "_index": "index1",
        "_id": "7",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 1,
          "failed": 0
        },
        "_seq_no": 0,
        "_primary_term": 1,
        "status": 201
      }
    }
  ]
}
An example response from `POST /_bulk?filter_path=items.*.error`, which returns only information about failed operations.
{
  "items": [
    {
      "update": {
        "error": {
          "type": "document_missing_exception",
          "reason": "[5]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    },
    {
      "update": {
        "error": {
          "type": "document_missing_exception",
          "reason": "[6]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    }
  ]
}

Bulk index or delete documents Generally available

POST /{index}/_bulk

Perform multiple index, create, delete, and update actions in a single request. This reduces overhead and can greatly increase indexing speed.

If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:

  • To use the create action, you must have the create_doc, create, index, or write index privilege. Data streams support only the create action.
  • To use the index action, you must have the create, index, or write index privilege.
  • To use the delete action, you must have the delete or write index privilege.
  • To use the update action, you must have the index or write index privilege.
  • To automatically create a data stream or index with a bulk API request, you must have the auto_configure, create_index, or manage index privilege.
  • To make the result of a bulk operation visible to search using the refresh parameter, you must have the maintenance or manage index privilege.

Automatic data stream creation requires a matching index template with data stream enabled.

The actions are specified in the request body using a newline delimited JSON (NDJSON) structure:

action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n

The index and create actions expect a source on the next line and have the same semantics as the op_type parameter in the standard index API. A create action fails if a document with the same ID already exists in the target An index action adds or replaces a document as necessary.

NOTE: Data streams support only the create action. To update or delete a document in a data stream, you must target the backing index containing the document.

An update action expects that the partial doc, upsert, and script and its options are specified on the next line.

A delete action does not expect a source on the next line and has the same semantics as the standard delete API.

NOTE: The final line of data must end with a newline character (\n). Each newline character may be preceded by a carriage return (\r). When sending NDJSON data to the _bulk endpoint, use a Content-Type header of application/json or application/x-ndjson. Because this format uses literal newline characters (\n) as delimiters, make sure that the JSON actions and sources are not pretty printed.

If you provide a target in the request path, it is used for any actions that don't explicitly specify an _index argument.

A note on the format: the idea here is to make processing as fast as possible. As some of the actions are redirected to other shards on other nodes, only action_meta_data is parsed on the receiving node side.

Client libraries using this protocol should try and strive to do something similar on the client side, and reduce buffering as much as possible.

There is no "correct" number of actions to perform in a single bulk request. Experiment with different settings to find the optimal size for your particular workload. Note that Elasticsearch limits the maximum size of a HTTP request to 100mb by default so clients must ensure that no request exceeds this size. It is not possible to index a single document that exceeds the size limit, so you must pre-process any such documents into smaller pieces before sending them to Elasticsearch. For instance, split documents into pages or chapters before indexing them, or store raw binary data in a system outside Elasticsearch and replace the raw data with a link to the external system in the documents that you send to Elasticsearch.

Client suppport for bulk requests

Some of the officially supported clients provide helpers to assist with bulk requests and reindexing:

  • Go: Check out esutil.BulkIndexer
  • Perl: Check out Search::Elasticsearch::Client::5_0::Bulk and Search::Elasticsearch::Client::5_0::Scroll
  • Python: Check out elasticsearch.helpers.*
  • JavaScript: Check out client.helpers.*
  • .NET: Check out BulkAllObservable
  • PHP: Check out bulk indexing.

Submitting bulk requests with cURL

If you're providing text file input to curl, you must use the --data-binary flag instead of plain -d. The latter doesn't preserve newlines. For example:

$ cat requests
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
$ curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests"; echo
{"took":7, "errors": false, "items":[{"index":{"_index":"test","_id":"1","_version":1,"result":"created","forced_refresh":false}}]}

Optimistic concurrency control

Each index and delete action within a bulk API call may include the if_seq_no and if_primary_term parameters in their respective action and meta data lines. The if_seq_no and if_primary_term parameters control how operations are run, based on the last modification to existing documents. See Optimistic concurrency control for more details.

Versioning

Each bulk item can include the version value using the version field. It automatically follows the behavior of the index or delete operation based on the _version mapping. It also support the version_type.

Routing

Each bulk item can include the routing value using the routing field. It automatically follows the behavior of the index or delete operation based on the _routing mapping.

NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.

Wait for active shards

When making bulk calls, you can set the wait_for_active_shards parameter to require a minimum number of shard copies to be active before starting to process the bulk request.

Refresh

Control when the changes made by this request are visible to search.

NOTE: Only the shards that receive the bulk request will be affected by refresh. Imagine a _bulk?refresh=wait_for request with three documents in it that happen to be routed to different shards in an index with five shards. The request will only wait for those three shards to refresh. The other two shards that make up the index do not participate in the _bulk request at all.

Path parameters

  • index string Required

    The name of the data stream, index, or index alias to perform bulk actions on.

Query parameters

  • include_source_on_error boolean

    True or false if to include the document source in the error message in case of parsing errors.

  • list_executed_pipelines boolean

    If true, the response will include the ingest pipelines that were run for each index or create.

  • pipeline string

    The pipeline identifier to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter.

  • refresh string

    If true, Elasticsearch refreshes the affected shards to make this operation visible to search. If wait_for, wait for a refresh to make this operation visible to search. If false, do nothing with refreshes. Valid values: true, false, wait_for.

    Values are true, false, or wait_for.

  • routing string

    A custom value that is used to route operations to a specific shard.

  • _source boolean | string | array[string]

    Indicates whether to return the _source field (true or false) or contains a list of fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter. If the _source parameter is false, this parameter is ignored.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter. If the _source parameter is false, this parameter is ignored.

  • timeout string

    The period each action waits for the following operations: automatic index creation, dynamic mapping updates, and waiting for active shards. The default is 1m (one minute), which guarantees Elasticsearch waits for at least the timeout before failing. The actual wait time could be longer, particularly when multiple waits occur.

    Values are -1 or 0.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default is 1, which waits for each primary shard to be active.

    Values are all or index-setting.

  • require_alias boolean

    If true, the request's actions must target an index alias.

  • require_data_stream boolean

    If true, the request's actions must target a data stream (existing or to be created).

application/json

Body object Required

One of:
  • index object
    Hide index attributes Show index attributes object
    • _id string
    • _index string
    • routing string
    • if_primary_term number
    • if_seq_no number
    • version number
    • version_type string

      Values are internal, external, external_gte, or force.

    • dynamic_templates object

      A map from the full name of fields to the name of dynamic templates. It defaults to an empty map. If a name matches a dynamic template, that template will be applied regardless of other match predicates defined in the template. If a field is already defined in the mapping, then this parameter won't be used.

      Hide dynamic_templates attribute Show dynamic_templates attribute object
      • * string Additional properties
    • pipeline string

      The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter.

    • require_alias boolean

      If true, the request's actions must target an index alias.

  • create object
    Hide create attributes Show create attributes object
    • _id string
    • _index string
    • routing string
    • if_primary_term number
    • if_seq_no number
    • version number
    • version_type string

      Values are internal, external, external_gte, or force.

    • dynamic_templates object

      A map from the full name of fields to the name of dynamic templates. It defaults to an empty map. If a name matches a dynamic template, that template will be applied regardless of other match predicates defined in the template. If a field is already defined in the mapping, then this parameter won't be used.

      Hide dynamic_templates attribute Show dynamic_templates attribute object
      • * string Additional properties
    • pipeline string

      The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter.

    • require_alias boolean

      If true, the request's actions must target an index alias.

  • update object
    Hide update attributes Show update attributes object
    • _id string
    • _index string
    • routing string
    • if_primary_term number
    • if_seq_no number
    • version number
    • version_type string

      Values are internal, external, external_gte, or force.

    • require_alias boolean

      If true, the request's actions must target an index alias.

    • retry_on_conflict number

      The number of times an update should be retried in the case of a version conflict.

  • delete object
    Hide delete attributes Show delete attributes object
    • _id string
    • _index string
    • routing string
    • if_primary_term number
    • if_seq_no number
    • version number
    • version_type string

      Values are internal, external, external_gte, or force.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • errors boolean Required

      If true, one or more of the operations in the bulk request did not complete successfully.

    • items array[object] Required

      The result of each operation in the bulk request, in the order they were submitted.

      Hide items attribute Show items attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • _id string | null

          The document ID associated with the operation.

        • _index string Required

          The name of the index associated with the operation. If the operation targeted a data stream, this is the backing index into which the document was written.

        • status number Required

          The HTTP status code returned for the operation.

        • failure_store string

          Values are not_applicable_or_unknown, used, not_enabled, or failed.

        • error object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide error attributes Show error attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • _primary_term number

          The primary term assigned to the document for the operation. This property is returned only for successful operations.

        • result string

          The result of the operation. Successful values are created, deleted, and updated.

        • _seq_no number
        • _shards object
          Hide _shards attributes Show _shards attributes object
          • failed number Required
          • successful number Required
          • total number Required
          • failures array[object]
          • skipped number
        • _version number
        • forced_refresh boolean
        • get object
          Hide get attributes Show get attributes object
          • fields object
            Hide fields attribute Show fields attribute object
            • * object Additional properties
          • found boolean Required
          • _seq_no number
          • _primary_term number
          • _routing string
          • _source object
            Hide _source attribute Show _source attribute object
            • * object Additional properties
    • took number Required

      The length of time, in milliseconds, it took to process the bulk request.

    • ingest_took number
POST /{index}/_bulk
curl \
 --request POST 'http://api.example.com/{index}/_bulk' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{ \"index\" : { \"_index\" : \"test\", \"_id\" : \"1\" } }\n{ \"field1\" : \"value1\" }\n{ \"delete\" : { \"_index\" : \"test\", \"_id\" : \"2\" } }\n{ \"create\" : { \"_index\" : \"test\", \"_id\" : \"3\" } }\n{ \"field1\" : \"value3\" }\n{ \"update\" : {\"_id\" : \"1\", \"_index\" : \"test\"} }\n{ \"doc\" : {\"field2\" : \"value2\"} }"'
Run `POST _bulk` to perform multiple operations.
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_id" : "2" } }
{ "create" : { "_index" : "test", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }
When you run `POST _bulk` and use the `update` action, you can use `retry_on_conflict` as a field in the action itself (not in the extra payload line) to specify how many times an update should be retried in the case of a version conflict.
{ "update" : {"_id" : "1", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_index" : "index1", "retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}
To return only information about failed operations, run `POST /_bulk?filter_path=items.*.error`.
{ "update": {"_id": "5", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "update": {"_id": "6", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "create": {"_id": "7", "_index": "index1"} }
{ "my_field": "foo" }
Run `POST /_bulk` to perform a bulk request that consists of index and create actions with the `dynamic_templates` parameter. The bulk request creates two new fields `work_location` and `home_location` with type `geo_point` according to the `dynamic_templates` parameter. However, the `raw_location` field is created using default dynamic mapping rules, as a text field in that case since it is supplied as a string in the JSON document.
{ "index" : { "_index" : "my_index", "_id" : "1", "dynamic_templates": {"work_location": "geo_point"}} }
{ "field" : "value1", "work_location": "41.12,-71.34", "raw_location": "41.12,-71.34"}
{ "create" : { "_index" : "my_index", "_id" : "2", "dynamic_templates": {"home_location": "geo_point"}} }
{ "field" : "value2", "home_location": "41.12,-71.34"}
Response examples (200)
{
   "took": 30,
   "errors": false,
   "items": [
      {
         "index": {
            "_index": "test",
            "_id": "1",
            "_version": 1,
            "result": "created",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "status": 201,
            "_seq_no" : 0,
            "_primary_term": 1
         }
      },
      {
         "delete": {
            "_index": "test",
            "_id": "2",
            "_version": 1,
            "result": "not_found",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "status": 404,
            "_seq_no" : 1,
            "_primary_term" : 2
         }
      },
      {
         "create": {
            "_index": "test",
            "_id": "3",
            "_version": 1,
            "result": "created",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "status": 201,
            "_seq_no" : 2,
            "_primary_term" : 3
         }
      },
      {
         "update": {
            "_index": "test",
            "_id": "1",
            "_version": 2,
            "result": "updated",
            "_shards": {
                "total": 2,
                "successful": 1,
                "failed": 0
            },
            "status": 200,
            "_seq_no" : 3,
            "_primary_term" : 4
         }
      }
   ]
}
If you run `POST /_bulk` with operations that update non-existent documents, the operations cannot complete successfully. The API returns a response with an `errors` property value `true`. The response also includes an error object for any failed operations. The error object contains additional information about the failure, such as the error type and reason.
{
  "took": 486,
  "errors": true,
  "items": [
    {
      "update": {
        "_index": "index1",
        "_id": "5",
        "status": 404,
        "error": {
          "type": "document_missing_exception",
          "reason": "[5]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    },
    {
      "update": {
        "_index": "index1",
        "_id": "6",
        "status": 404,
        "error": {
          "type": "document_missing_exception",
          "reason": "[6]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    },
    {
      "create": {
        "_index": "index1",
        "_id": "7",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 1,
          "failed": 0
        },
        "_seq_no": 0,
        "_primary_term": 1,
        "status": 201
      }
    }
  ]
}
An example response from `POST /_bulk?filter_path=items.*.error`, which returns only information about failed operations.
{
  "items": [
    {
      "update": {
        "error": {
          "type": "document_missing_exception",
          "reason": "[5]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    },
    {
      "update": {
        "error": {
          "type": "document_missing_exception",
          "reason": "[6]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    }
  ]
}












Create or update a document in an index Generally available

PUT /{index}/_doc/{id}

Add a JSON document to the specified data stream or index and make it searchable. If the target is an index and the document already exists, the request updates the document and increments its version.

NOTE: You cannot use this API to send update requests for existing documents in a data stream.

If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:

  • To add or overwrite a document using the PUT /<target>/_doc/<_id> request format, you must have the create, index, or write index privilege.
  • To add a document using the POST /<target>/_doc/ request format, you must have the create_doc, create, index, or write index privilege.
  • To automatically create a data stream or index with this API request, you must have the auto_configure, create_index, or manage index privilege.

Automatic data stream creation requires a matching index template with data stream enabled.

NOTE: Replica shards might not all be started when an indexing operation returns successfully. By default, only the primary is required. Set wait_for_active_shards to change this default behavior.

Automatically create data streams and indices

If the request's target doesn't exist and matches an index template with a data_stream definition, the index operation automatically creates the data stream.

If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.

NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.

If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.

Automatic index creation is controlled by the action.auto_create_index setting. If it is true, any index can be created automatically. You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false to turn off automatic index creation entirely. Specify a comma-separated list of patterns you want to allow or prefix each pattern with + or - to indicate whether it should be allowed or blocked. When a list is specified, the default behaviour is to disallow.

NOTE: The action.auto_create_index setting affects the automatic creation of indices only. It does not affect the creation of data streams.

Optimistic concurrency control

Index operations can be made conditional and only be performed if the last modification to the document was assigned the sequence number and primary term specified by the if_seq_no and if_primary_term parameters. If a mismatch is detected, the operation will result in a VersionConflictException and a status code of 409.

Routing

By default, shard placement — or routing — is controlled by using a hash of the document's ID value. For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing parameter.

When setting up explicit mapping, you can also use the _routing field to direct the index operation to extract the routing value from the document itself. This does come at the (very minimal) cost of an additional document parsing pass. If the _routing mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.

NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.

Distributed

The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.

Active shards

To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation. If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs. By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards is 1). This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards. To alter this behavior per operation, use the wait_for_active_shards request parameter.

Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas+1). Specifying a negative value or a number greater than the number of shard copies will throw an error.

For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes). If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding. This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data. If wait_for_active_shards is set on the request to 3 (and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding. This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard. However, if you set wait_for_active_shards to all (or to 4, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index. The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.

It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts. After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary. The _shards section of the API response reveals the number of shard copies on which replication succeeded and failed.

No operation (noop) updates

When updating a document by using this API, a new version of the document is always created even if the document hasn't changed. If this isn't acceptable use the _update API with detect_noop set to true. The detect_noop option isn't available on this API because it doesn’t fetch the old source and isn't able to compare it against the new source.

There isn't a definitive rule for when noop updates aren't acceptable. It's a combination of lots of factors like how frequently your data source sends updates that are actually noops and how many queries per second Elasticsearch runs on the shard receiving the updates.

Versioning

Each indexed document is given a version number. By default, internal versioning is used that starts at 1 and increments with each update, deletes included. Optionally, the version number can be set to an external value (for example, if maintained in a database). To enable this functionality, version_type should be set to external. The value provided must be a numeric, long value greater than or equal to 0, and less than around 9.2e+18.

NOTE: Versioning is completely real time, and is not affected by the near real time aspects of search operations. If no version is provided, the operation runs without any version checks.

When using the external version type, the system checks to see if the version number passed to the index request is greater than the version of the currently stored document. If true, the document will be indexed and the new version number used. If the value provided is less than or equal to the stored document's version number, a version conflict will occur and the index operation will fail. For example:

PUT my-index-000001/_doc/1?version=2&version_type=external
{
  "user": {
    "id": "elkbee"
  }
}

In this example, the operation will succeed since the supplied version of 2 is higher than the current document version of 1.
If the document was already updated and its version was set to 2 or higher, the indexing command will fail and result in a conflict (409 HTTP status code).

A nice side effect is that there is no need to maintain strict ordering of async indexing operations run as a result of changes to a source database, as long as version numbers from the source database are used.
Even the simple case of updating the Elasticsearch index using data from a database is simplified if external versioning is used, as only the latest version will be used if the index operations arrive out of order.

## Required authorization

* Index privileges: `index`
External documentation

Path parameters

  • index string Required

    The name of the data stream or index to target. If the target doesn't exist and matches the name or wildcard (*) pattern of an index template with a data_stream definition, this request creates the data stream. If the target doesn't exist and doesn't match a data stream template, this request creates the index. You can check for existing targets with the resolve index API.

  • id string Required

    A unique identifier for the document. To automatically generate a document ID, use the POST /<target>/_doc/ request format and omit this parameter.

Query parameters

  • if_primary_term number

    Only perform the operation if the document has this primary term.

  • if_seq_no number

    Only perform the operation if the document has this sequence number.

  • include_source_on_error boolean

    True or false if to include the document source in the error message in case of parsing errors.

  • op_type string

    Set to create to only index the document if it does not already exist (put if absent). If a document with the specified _id already exists, the indexing operation will fail. The behavior is the same as using the <index>/_create endpoint. If a document ID is specified, this paramater defaults to index. Otherwise, it defaults to create. If the request targets a data stream, an op_type of create is required.

    Values are index or create.

  • pipeline string

    The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, then setting the value to _none disables the default ingest pipeline for this request. If a final pipeline is configured it will always run, regardless of the value of this parameter.

  • refresh string

    If true, Elasticsearch refreshes the affected shards to make this operation visible to search. If wait_for, it waits for a refresh to make this operation visible to search. If false, it does nothing with refreshes.

    Values are true, false, or wait_for.

  • routing string

    A custom value that is used to route operations to a specific shard.

  • timeout string

    The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards.

    This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.

    Values are -1 or 0.

  • version number

    An explicit version number for concurrency control. It must be a non-negative long number.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default value of 1 means it waits for each primary shard to be active.

    Values are all or index-setting.

  • require_alias boolean

    If true, the destination must be an index alias.

application/json

Body Required

object object

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _id string Required
    • _index string Required
    • _primary_term number

      The primary term assigned to the document for the indexing operation.

    • result string Required

      Values are created, updated, deleted, not_found, or noop.

    • _seq_no number
    • _shards object Required
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number Required
        • status string
      • skipped number
    • _version number Required
    • forced_refresh boolean
PUT /{index}/_doc/{id}
curl \
 --request PUT 'http://api.example.com/{index}/_doc/{id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"@timestamp\": \"2099-11-15T13:12:00\",\n  \"message\": \"GET /search HTTP/1.1 200 1070000\",\n  \"user\": {\n    \"id\": \"kimchy\"\n  }\n}"'
Request examples
Run `POST my-index-000001/_doc/` to index a document. When you use the `POST /<target>/_doc/` request format, the `op_type` is automatically set to `create` and the index operation generates a unique ID for the document.
{
  "@timestamp": "2099-11-15T13:12:00",
  "message": "GET /search HTTP/1.1 200 1070000",
  "user": {
    "id": "kimchy"
  }
}
Run `PUT my-index-000001/_doc/1` to insert a JSON document into the `my-index-000001` index with an `_id` of 1.
{
  "@timestamp": "2099-11-15T13:12:00",
  "message": "GET /search HTTP/1.1 200 1070000",
  "user": {
    "id": "kimchy"
  }
}
Response examples (200)
A successful response from `POST my-index-000001/_doc/`, which contains an automated document ID.
{
  "_shards": {
    "total": 2,
    "failed": 0,
    "successful": 2
  },
  "_index": "my-index-000001",
  "_id": "W0tpsmIBdwcYyG50zbta",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "result": "created"
}
A successful response from `PUT my-index-000001/_doc/1`.
{
  "_shards": {
    "total": 2,
    "failed": 0,
    "successful": 2
  },
  "_index": "my-index-000001",
  "_id": "1",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "result": "created"
}




Delete a document Generally available

DELETE /{index}/_doc/{id}

Remove a JSON document from the specified index.

NOTE: You cannot send deletion requests directly to a data stream. To delete a document in a data stream, you must target the backing index containing the document.

Optimistic concurrency control

Delete operations can be made conditional and only be performed if the last modification to the document was assigned the sequence number and primary term specified by the if_seq_no and if_primary_term parameters. If a mismatch is detected, the operation will result in a VersionConflictException and a status code of 409.

Versioning

Each document indexed is versioned. When deleting a document, the version can be specified to make sure the relevant document you are trying to delete is actually being deleted and it has not changed in the meantime. Every write operation run on a document, deletes included, causes its version to be incremented. The version number of a deleted document remains available for a short time after deletion to allow for control of concurrent operations. The length of time for which a deleted document's version remains available is determined by the index.gc_deletes index setting.

Routing

If routing is used during indexing, the routing value also needs to be specified to delete a document.

If the _routing mapping is set to required and no routing value is specified, the delete API throws a RoutingMissingException and rejects the request.

For example:

DELETE /my-index-000001/_doc/1?routing=shard-1

This request deletes the document with ID 1, but it is routed based on the user. The document is not deleted if the correct routing is not specified.

Distributed

The delete operation gets hashed into a specific shard ID. It then gets redirected into the primary shard within that ID group and replicated (if needed) to shard replicas within that ID group.

Required authorization

  • Index privileges: delete

Path parameters

  • index string Required

    The name of the target index.

  • id string Required

    A unique identifier for the document.

Query parameters

  • if_primary_term number

    Only perform the operation if the document has this primary term.

  • if_seq_no number

    Only perform the operation if the document has this sequence number.

  • refresh string

    If true, Elasticsearch refreshes the affected shards to make this operation visible to search. If wait_for, it waits for a refresh to make this operation visible to search. If false, it does nothing with refreshes.

    Values are true, false, or wait_for.

  • routing string

    A custom value used to route operations to a specific shard.

  • timeout string

    The period to wait for active shards.

    This parameter is useful for situations where the primary shard assigned to perform the delete operation might not be available when the delete operation runs. Some reasons for this might be that the primary shard is currently recovering from a store or undergoing relocation. By default, the delete operation will wait on the primary shard to become available for up to 1 minute before failing and responding with an error.

    Values are -1 or 0.

  • version number

    An explicit version number for concurrency control. It must match the current version of the document for the request to succeed.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

  • wait_for_active_shards number | string

    The minimum number of shard copies that must be active before proceeding with the operation. You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default value of 1 means it waits for each primary shard to be active.

    Values are all or index-setting.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _id string Required
    • _index string Required
    • _primary_term number

      The primary term assigned to the document for the indexing operation.

    • result string Required

      Values are created, updated, deleted, not_found, or noop.

    • _seq_no number
    • _shards object Required
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number Required
        • status string
      • skipped number
    • _version number Required
    • forced_refresh boolean
DELETE /{index}/_doc/{id}
curl \
 --request DELETE 'http://api.example.com/{index}/_doc/{id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `DELETE /my-index-000001/_doc/1`, which deletes the JSON document 1 from the `my-index-000001` index.
{
  "_shards": {
    "total": 2,
    "failed": 0,
    "successful": 2
  },
  "_index": "my-index-000001",
  "_id": "1",
  "_version": 2,
  "_primary_term": 1,
  "_seq_no": 5,
  "result": "deleted"
}

Check a document Generally available

HEAD /{index}/_doc/{id}

Verify that a document exists. For example, check to see if a document with the _id 0 exists:

HEAD my-index-000001/_doc/0

If the document exists, the API returns a status code of 200 - OK. If the document doesn’t exist, the API returns 404 - Not Found.

Versioning support

You can use the version parameter to check the document only if its current version is equal to the specified one.

Internally, Elasticsearch has marked the old document as deleted and added an entirely new document. The old version of the document doesn't disappear immediately, although you won't be able to access it. Elasticsearch cleans up deleted documents in the background as you continue to index more data.

Path parameters

  • index string Required

    A comma-separated list of data streams, indices, and aliases. It supports wildcards (*).

  • id string Required

    A unique document identifier.

Query parameters

  • preference string

    The node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.

    If it is set to _local, the operation will prefer to be run on a local allocated shard when possible. If it is set to a custom value, the value is used to guarantee that the same shards will be used for the same custom value. This can help with "jumping values" when hitting different shards in different refresh states. A sample value can be something like the web session ID or the user name.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • refresh boolean

    If true, the request refreshes the relevant shards before retrieving the document. Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).

  • routing string

    A custom value used to route operations to a specific shard.

  • _source boolean | string | array[string]

    Indicates whether to return the _source field (true or false) or lists the fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter. If the _source parameter is false, this parameter is ignored.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter. If the _source parameter is false, this parameter is ignored.

  • stored_fields string | array[string]

    A comma-separated list of stored fields to return as part of a hit. If no fields are specified, no stored fields are included in the response. If this field is specified, the _source parameter defaults to false.

  • version number

    Explicit version number for concurrency control. The specified version must match the current version of the document for the request to succeed.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

Responses

  • 200 application/json
HEAD /{index}/_doc/{id}
curl \
 --request HEAD 'http://api.example.com/{index}/_doc/{id}' \
 --header "Authorization: $API_KEY"

Delete documents Generally available; Added in 5.0.0

POST /{index}/_delete_by_query

Deletes documents that match the specified query.

If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or alias:

  • read
  • delete or write

You can specify the query criteria in the request URI or the request body using the same syntax as the search API. When you submit a delete by query request, Elasticsearch gets a snapshot of the data stream or index when it begins processing the request and deletes matching documents using internal versioning. If a document changes between the time that the snapshot is taken and the delete operation is processed, it results in a version conflict and the delete operation fails.

NOTE: Documents with a version equal to 0 cannot be deleted using delete by query because internal versioning does not support 0 as a valid version number.

While processing a delete by query request, Elasticsearch performs multiple search requests sequentially to find all of the matching documents to delete. A bulk delete request is performed for each batch of matching documents. If a search or bulk request is rejected, the requests are retried up to 10 times, with exponential back off. If the maximum retry limit is reached, processing halts and all failed requests are returned in the response. Any delete requests that completed successfully still stick, they are not rolled back.

You can opt to count version conflicts instead of halting and returning by setting conflicts to proceed. Note that if you opt to count version conflicts the operation could attempt to delete more documents from the source than max_docs until it has successfully deleted max_docs documents, or it has gone through every document in the source query.

Throttling delete requests

To control the rate at which delete by query issues batches of delete operations, you can set requests_per_second to any positive decimal number. This pads each batch with a wait time to throttle the rate. Set requests_per_second to -1 to disable throttling.

Throttling uses a wait time between batches so that the internal scroll requests can be given a timeout that takes the request padding into account. The padding time is the difference between the batch size divided by the requests_per_second and the time spent writing. By default the batch size is 1000, so if requests_per_second is set to 500:

target_time = 1000 / 500 per second = 2 seconds
wait_time = target_time - write_time = 2 seconds - .5 seconds = 1.5 seconds

Since the batch is issued as a single _bulk request, large batch sizes cause Elasticsearch to create many requests and wait before starting the next set. This is "bursty" instead of "smooth".

Slicing

Delete by query supports sliced scroll to parallelize the delete process. This can improve efficiency and provide a convenient way to break the request down into smaller parts.

Setting slices to auto lets Elasticsearch choose the number of slices to use. This setting will use one slice per shard, up to a certain limit. If there are multiple source data streams or indices, it will choose the number of slices based on the index or backing index with the smallest number of shards. Adding slices to the delete by query operation creates sub-requests which means it has some quirks:

  • You can see these requests in the tasks APIs. These sub-requests are "child" tasks of the task for the request with slices.
  • Fetching the status of the task for the request with slices only contains the status of completed slices.
  • These sub-requests are individually addressable for things like cancellation and rethrottling.
  • Rethrottling the request with slices will rethrottle the unfinished sub-request proportionally.
  • Canceling the request with slices will cancel each sub-request.
  • Due to the nature of slices each sub-request won't get a perfectly even portion of the documents. All documents will be addressed, but some slices may be larger than others. Expect larger slices to have a more even distribution.
  • Parameters like requests_per_second and max_docs on a request with slices are distributed proportionally to each sub-request. Combine that with the earlier point about distribution being uneven and you should conclude that using max_docs with slices might not result in exactly max_docs documents being deleted.
  • Each sub-request gets a slightly different snapshot of the source data stream or index though these are all taken at approximately the same time.

If you're slicing manually or otherwise tuning automatic slicing, keep in mind that:

  • Query performance is most efficient when the number of slices is equal to the number of shards in the index or backing index. If that number is large (for example, 500), choose a lower number as too many slices hurts performance. Setting slices higher than the number of shards generally does not improve efficiency and adds overhead.
  • Delete performance scales linearly across available resources with the number of slices.

Whether query or delete performance dominates the runtime depends on the documents being reindexed and cluster resources.

Cancel a delete by query operation

Any delete by query can be canceled using the task cancel API. For example:

POST _tasks/r1A2WoRbTwKZ516z6NEs5A:36619/_cancel

The task ID can be found by using the get tasks API.

Cancellation should happen quickly but might take a few seconds. The get task status API will continue to list the delete by query task until this task checks that it has been cancelled and terminates itself.

Required authorization

  • Index privileges: read,delete

Path parameters

  • index string | array[string] Required

    A comma-separated list of data streams, indices, and aliases to search. It supports wildcards (*). To search all data streams or indices, omit this parameter or use * or _all.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • analyzer string

    Analyzer to use for the query string. This parameter can be used only when the q query string parameter is specified.

  • analyze_wildcard boolean

    If true, wildcard and prefix queries are analyzed. This parameter can be used only when the q query string parameter is specified.

  • conflicts string

    What to do if delete by query hits version conflicts: abort or proceed.

    Values are abort or proceed.

  • default_operator string

    The default operator for query string query: AND or OR. This parameter can be used only when the q query string parameter is specified.

    Values are and, AND, or, or OR.

  • df string

    The field to use as default where no field prefix is given in the query string. This parameter can be used only when the q query string parameter is specified.

  • expand_wildcards string | array[string]

    The type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. It supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • from number

    Skips the specified number of documents.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • lenient boolean

    If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored. This parameter can be used only when the q query string parameter is specified.

  • max_docs number

    The maximum number of documents to process. Defaults to all documents. When set to a value less then or equal to scroll_size, a scroll will not be used to retrieve the results for the operation.

  • preference string

    The node or shard the operation should be performed on. It is random by default.

  • refresh boolean

    If true, Elasticsearch refreshes all shards involved in the delete by query after the request completes. This is different than the delete API's refresh parameter, which causes just the shard that received the delete request to be refreshed. Unlike the delete API, it does not support wait_for.

  • request_cache boolean

    If true, the request cache is used for this request. Defaults to the index-level setting.

  • requests_per_second number

    The throttle for this request in sub-requests per second.

  • routing string

    A custom value used to route operations to a specific shard.

  • q string

    A query in the Lucene query string syntax.

  • scroll string

    The period to retain the search context for scrolling.

    Values are -1 or 0.

  • scroll_size number

    The size of the scroll request that powers the operation.

  • search_timeout string

    The explicit timeout for each search request. It defaults to no timeout.

    Values are -1 or 0.

  • search_type string

    The type of the search operation. Available options include query_then_fetch and dfs_query_then_fetch.

    Values are query_then_fetch or dfs_query_then_fetch.

  • slices number | string

    The number of slices this task should be divided into.

    Value is auto.

  • sort array[string]

    A comma-separated list of <field>:<direction> pairs.

  • stats array[string]

    The specific tag of the request for logging and statistical purposes.

  • terminate_after number

    The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.

    Use with caution. Elasticsearch applies this parameter to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this parameter for requests that target data streams with backing indices across multiple data tiers.

  • timeout string

    The period each deletion request waits for active shards.

    Values are -1 or 0.

  • version boolean

    If true, returns the document version as part of a hit.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The timeout value controls how long each write request waits for unavailable shards to become available.

    Values are all or index-setting.

  • wait_for_completion boolean

    If true, the request blocks until the operation is complete. If false, Elasticsearch performs some preflight checks, launches the request, and returns a task you can use to cancel or get the status of the task. Elasticsearch creates a record of this task as a document at .tasks/task/${taskId}. When you are done with a task, you should delete the task document so Elasticsearch can reclaim the space.

application/json

Body Required

  • max_docs number

    The maximum number of documents to delete.

  • query object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation
  • slice object
    Hide slice attributes Show slice attributes object
    • field string

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • id string Required
    • max number Required

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • batches number

      The number of scroll responses pulled back by the delete by query.

    • deleted number

      The number of documents that were successfully deleted.

    • failures array[object]

      An array of failures if there were any unrecoverable errors during the process. If this array is not empty, the request ended abnormally because of those failures. Delete by query is implemented using batches and any failures cause the entire process to end but all failures in the current batch are collected into the array. You can use the conflicts option to prevent reindex from ending on version conflicts.

      Hide failures attributes Show failures attributes object
      • cause object Required

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide cause attributes Show cause attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • id string Required
      • index string Required
      • status number Required
    • noops number

      This field is always equal to zero for delete by query. It exists only so that delete by query, update by query, and reindex APIs return responses with the same structure.

    • requests_per_second number

      The number of requests per second effectively run during the delete by query.

    • retries object
      Hide retries attributes Show retries attributes object
      • bulk number Required

        The number of bulk actions retried.

    • slice_id number
    • task string | number

    • throttled string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • throttled_millis number

      Time unit for milliseconds

    • throttled_until string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • throttled_until_millis number

      Time unit for milliseconds

    • timed_out boolean

      If true, some requests run during the delete by query operation timed out.

    • took number

      Time unit for milliseconds

    • total number

      The number of documents that were successfully processed.

    • version_conflicts number

      The number of version conflicts that the delete by query hit.

POST /{index}/_delete_by_query
curl \
 --request POST 'http://api.example.com/{index}/_delete_by_query' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"query\": {\n    \"match_all\": {}\n  }\n}"'
Run `POST /my-index-000001,my-index-000002/_delete_by_query` to delete all documents from multiple data streams or indices.
{
  "query": {
    "match_all": {}
  }
}
Run `POST my-index-000001/_delete_by_query` to delete a document by using a unique attribute.
{
  "query": {
    "term": {
      "user.id": "kimchy"
    }
  },
  "max_docs": 1
}
Run `POST my-index-000001/_delete_by_query` to slice a delete by query manually. Provide a slice ID and total number of slices.
{
  "slice": {
    "id": 0,
    "max": 2
  },
  "query": {
    "range": {
      "http.response.bytes": {
        "lt": 2000000
      }
    }
  }
}
Run `POST my-index-000001/_delete_by_query?refresh&slices=5` to let delete by query automatically parallelize using sliced scroll to slice on `_id`. The `slices` query parameter value specifies the number of slices to use.
{
  "query": {
    "range": {
      "http.response.bytes": {
        "lt": 2000000
      }
    }
  }
}
Response examples (200)
A successful response from `POST /my-index-000001/_delete_by_query`.
{
  "took" : 147,
  "timed_out": false,
  "total": 119,
  "deleted": 119,
  "batches": 1,
  "version_conflicts": 0,
  "noops": 0,
  "retries": {
    "bulk": 0,
    "search": 0
  },
  "throttled_millis": 0,
  "requests_per_second": -1.0,
  "throttled_until_millis": 0,
  "failures" : [ ]
}




Get a document's source Generally available

GET /{index}/_source/{id}

Get the source of a document. For example:

GET my-index-000001/_source/1

You can use the source filtering parameters to control which parts of the _source are returned:

GET my-index-000001/_source/1/?_source_includes=*.id&_source_excludes=entities

Required authorization

  • Index privileges: read
External documentation

Path parameters

  • index string Required

    The name of the index that contains the document.

  • id string Required

    A unique document identifier.

Query parameters

  • preference string

    The node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • refresh boolean

    If true, the request refreshes the relevant shards before retrieving the document. Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).

  • routing string

    A custom value used to route operations to a specific shard.

  • _source boolean | string | array[string]

    Indicates whether to return the _source field (true or false) or lists the fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude in the response.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response.

  • stored_fields string | array[string]

    A comma-separated list of stored fields to return as part of a hit.

  • version number

    The version number for concurrency control. It must match the current version of the document for the request to succeed.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

Responses

  • 200 application/json
GET /{index}/_source/{id}
curl \
 --request GET 'http://api.example.com/{index}/_source/{id}' \
 --header "Authorization: $API_KEY"

Check for a document source Generally available; Added in 5.4.0

HEAD /{index}/_source/{id}

Check whether a document source exists in an index. For example:

HEAD my-index-000001/_source/1

A document's source is not available if it is disabled in the mapping.

Required authorization

  • Index privileges: read
External documentation

Path parameters

  • index string Required

    A comma-separated list of data streams, indices, and aliases. It supports wildcards (*).

  • id string Required

    A unique identifier for the document.

Query parameters

  • preference string

    The node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • refresh boolean

    If true, the request refreshes the relevant shards before retrieving the document. Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).

  • routing string

    A custom value used to route operations to a specific shard.

  • _source boolean | string | array[string]

    Indicates whether to return the _source field (true or false) or lists the fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude in the response.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response.

  • version number

    The version number for concurrency control. It must match the current version of the document for the request to succeed.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

Responses

  • 200 application/json
HEAD /{index}/_source/{id}
curl \
 --request HEAD 'http://api.example.com/{index}/_source/{id}' \
 --header "Authorization: $API_KEY"




Get multiple documents Generally available; Added in 1.3.0

GET /_mget

Get multiple JSON documents by ID from one or more indices. If you specify an index in the request URI, you only need to specify the document IDs in the request body. To ensure fast responses, this multi get (mget) API responds with partial results if one or more shards fail.

Filter source fields

By default, the _source field is returned for every document (if stored). Use the _source and _source_include or source_exclude attributes to filter what fields are returned for a particular document. You can include the _source, _source_includes, and _source_excludes query parameters in the request URI to specify the defaults to use when there are no per-document instructions.

Get stored fields

Use the stored_fields attribute to specify the set of stored fields you want to retrieve. Any requested fields that are not stored are ignored. You can include the stored_fields query parameter in the request URI to specify the defaults to use when there are no per-document instructions.

Required authorization

  • Index privileges: read

Query parameters

  • preference string

    Specifies the node or shard the operation should be performed on. Random by default.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • refresh boolean

    If true, the request refreshes relevant shards before retrieving documents.

  • routing string

    Custom value used to route operations to a specific shard.

  • _source boolean | string | array[string]

    True or false to return the _source field or not, or a list of fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter. If the _source parameter is false, this parameter is ignored.

  • stored_fields string | array[string]

    If true, retrieves the document fields stored in the index rather than the document _source.

application/json

Body Required

  • docs array[object]

    The documents you want to retrieve. Required if no index is specified in the request URI.

    Hide docs attributes Show docs attributes object
    • _id string Required
    • _index string
    • routing string
    • _source boolean | object

      Defines how to fetch a source. Fetching can be disabled entirely, or the source can be filtered.

      One of:
    • stored_fields string | array[string]
    • version number
    • version_type string

      Values are internal, external, external_gte, or force.

  • ids string | array[string]

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required

      The response includes a docs array that contains the documents in the order specified in the request. The structure of the returned documents is similar to that returned by the get API. If there is a failure getting a particular document, the error is included in place of the document.

      One of:
      Hide attributes Show attributes
      • _index string Required
      • fields object

        If the stored_fields parameter is set to true and found is true, it contains the document fields stored in the index.

        Hide fields attribute Show fields attribute object
        • * object Additional properties
      • _ignored array[string]
      • found boolean Required

        Indicates whether the document exists.

      • _id string Required
      • _primary_term number

        The primary term assigned to the document for the indexing operation.

      • _routing string

        The explicit routing, if set.

      • _seq_no number
      • _source object

        If found is true, it contains the document data formatted in JSON. If the _source parameter is set to false or the stored_fields parameter is set to true, it is excluded.

      • _version number
GET /_mget
curl \
 --request GET 'http://api.example.com/_mget' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"docs\": [\n    {\n      \"_id\": \"1\"\n    },\n    {\n      \"_id\": \"2\"\n    }\n  ]\n}"'
Run `GET /my-index-000001/_mget`. When you specify an index in the request URI, only the document IDs are required in the request body.
{
  "docs": [
    {
      "_id": "1"
    },
    {
      "_id": "2"
    }
  ]
}
Run `GET /_mget`. This request sets `_source` to `false` for document 1 to exclude the source entirely. It retrieves `field3` and `field4` from document 2. It retrieves the `user` field from document 3 but filters out the `user.location` field.
{
  "docs": [
    {
      "_index": "test",
      "_id": "1",
      "_source": false
    },
    {
      "_index": "test",
      "_id": "2",
      "_source": [ "field3", "field4" ]
    },
    {
      "_index": "test",
      "_id": "3",
      "_source": {
        "include": [ "user" ],
        "exclude": [ "user.location" ]
      }
    }
  ]
}
Run `GET /_mget`. This request retrieves `field1` and `field2` from document 1 and `field3` and `field4` from document 2.
{
  "docs": [
    {
      "_index": "test",
      "_id": "1",
      "stored_fields": [ "field1", "field2" ]
    },
    {
      "_index": "test",
      "_id": "2",
      "stored_fields": [ "field3", "field4" ]
    }
  ]
}
Run `GET /_mget?routing=key1`. If routing is used during indexing, you need to specify the routing value to retrieve documents. This request fetches `test/_doc/2` from the shard corresponding to routing key `key1`. It fetches `test/_doc/1` from the shard corresponding to routing key `key2`.
{
  "docs": [
    {
      "_index": "test",
      "_id": "1",
      "routing": "key2"
    },
    {
      "_index": "test",
      "_id": "2"
    }
  ]
}








Get multiple documents Generally available; Added in 1.3.0

POST /{index}/_mget

Get multiple JSON documents by ID from one or more indices. If you specify an index in the request URI, you only need to specify the document IDs in the request body. To ensure fast responses, this multi get (mget) API responds with partial results if one or more shards fail.

Filter source fields

By default, the _source field is returned for every document (if stored). Use the _source and _source_include or source_exclude attributes to filter what fields are returned for a particular document. You can include the _source, _source_includes, and _source_excludes query parameters in the request URI to specify the defaults to use when there are no per-document instructions.

Get stored fields

Use the stored_fields attribute to specify the set of stored fields you want to retrieve. Any requested fields that are not stored are ignored. You can include the stored_fields query parameter in the request URI to specify the defaults to use when there are no per-document instructions.

Required authorization

  • Index privileges: read

Path parameters

  • index string Required

    Name of the index to retrieve documents from when ids are specified, or when a document in the docs array does not specify an index.

Query parameters

  • preference string

    Specifies the node or shard the operation should be performed on. Random by default.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • refresh boolean

    If true, the request refreshes relevant shards before retrieving documents.

  • routing string

    Custom value used to route operations to a specific shard.

  • _source boolean | string | array[string]

    True or false to return the _source field or not, or a list of fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter. If the _source parameter is false, this parameter is ignored.

  • stored_fields string | array[string]

    If true, retrieves the document fields stored in the index rather than the document _source.

application/json

Body Required

  • docs array[object]

    The documents you want to retrieve. Required if no index is specified in the request URI.

    Hide docs attributes Show docs attributes object
    • _id string Required
    • _index string
    • routing string
    • _source boolean | object

      Defines how to fetch a source. Fetching can be disabled entirely, or the source can be filtered.

      One of:
    • stored_fields string | array[string]
    • version number
    • version_type string

      Values are internal, external, external_gte, or force.

  • ids string | array[string]

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required

      The response includes a docs array that contains the documents in the order specified in the request. The structure of the returned documents is similar to that returned by the get API. If there is a failure getting a particular document, the error is included in place of the document.

      One of:
      Hide attributes Show attributes
      • _index string Required
      • fields object

        If the stored_fields parameter is set to true and found is true, it contains the document fields stored in the index.

        Hide fields attribute Show fields attribute object
        • * object Additional properties
      • _ignored array[string]
      • found boolean Required

        Indicates whether the document exists.

      • _id string Required
      • _primary_term number

        The primary term assigned to the document for the indexing operation.

      • _routing string

        The explicit routing, if set.

      • _seq_no number
      • _source object

        If found is true, it contains the document data formatted in JSON. If the _source parameter is set to false or the stored_fields parameter is set to true, it is excluded.

      • _version number
POST /{index}/_mget
curl \
 --request POST 'http://api.example.com/{index}/_mget' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"docs\": [\n    {\n      \"_id\": \"1\"\n    },\n    {\n      \"_id\": \"2\"\n    }\n  ]\n}"'
Run `GET /my-index-000001/_mget`. When you specify an index in the request URI, only the document IDs are required in the request body.
{
  "docs": [
    {
      "_id": "1"
    },
    {
      "_id": "2"
    }
  ]
}
Run `GET /_mget`. This request sets `_source` to `false` for document 1 to exclude the source entirely. It retrieves `field3` and `field4` from document 2. It retrieves the `user` field from document 3 but filters out the `user.location` field.
{
  "docs": [
    {
      "_index": "test",
      "_id": "1",
      "_source": false
    },
    {
      "_index": "test",
      "_id": "2",
      "_source": [ "field3", "field4" ]
    },
    {
      "_index": "test",
      "_id": "3",
      "_source": {
        "include": [ "user" ],
        "exclude": [ "user.location" ]
      }
    }
  ]
}
Run `GET /_mget`. This request retrieves `field1` and `field2` from document 1 and `field3` and `field4` from document 2.
{
  "docs": [
    {
      "_index": "test",
      "_id": "1",
      "stored_fields": [ "field1", "field2" ]
    },
    {
      "_index": "test",
      "_id": "2",
      "stored_fields": [ "field3", "field4" ]
    }
  ]
}
Run `GET /_mget?routing=key1`. If routing is used during indexing, you need to specify the routing value to retrieve documents. This request fetches `test/_doc/2` from the shard corresponding to routing key `key1`. It fetches `test/_doc/1` from the shard corresponding to routing key `key2`.
{
  "docs": [
    {
      "_index": "test",
      "_id": "1",
      "routing": "key2"
    },
    {
      "_index": "test",
      "_id": "2"
    }
  ]
}

Get multiple term vectors Generally available

GET /_mtermvectors

Get multiple term vectors with a single request. You can specify existing documents by index and ID or provide artificial documents in the body of the request. You can specify the index in the request body or request URI. The response contains a docs array with all the fetched termvectors. Each element has the structure provided by the termvectors API.

Artificial documents

You can also use mtermvectors to generate term vectors for artificial documents provided in the body of the request. The mapping used is determined by the specified _index.

Required authorization

  • Index privileges: read

Query parameters

  • ids array[string]

    A comma-separated list of documents ids. You must define ids as parameter or set "ids" or "docs" in the request body

  • fields string | array[string]

    A comma-separated list or wildcard expressions of fields to include in the statistics. It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.

  • field_statistics boolean

    If true, the response includes the document count, sum of document frequencies, and sum of total term frequencies.

  • offsets boolean

    If true, the response includes term offsets.

  • payloads boolean

    If true, the response includes term payloads.

  • positions boolean

    If true, the response includes term positions.

  • preference string

    The node or shard the operation should be performed on. It is random by default.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • routing string

    A custom value used to route operations to a specific shard.

  • term_statistics boolean

    If true, the response includes term frequency and document frequency.

  • version number

    If true, returns the document version as part of a hit.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

application/json

Body

  • docs array[object]

    An array of existing or artificial documents.

    Hide docs attributes Show docs attributes object
    • _id string
    • _index string
    • doc object

      An artificial document (a document not present in the index) for which you want to retrieve term vectors.

    • fields string | array[string]
    • field_statistics boolean

      If true, the response includes the document count, sum of document frequencies, and sum of total term frequencies.

    • filter object
      Hide filter attributes Show filter attributes object
      • max_doc_freq number

        Ignore words which occur in more than this many docs. Defaults to unbounded.

      • max_num_terms number

        The maximum number of terms that must be returned per field.

      • max_term_freq number

        Ignore words with more than this frequency in the source doc. It defaults to unbounded.

      • max_word_length number

        The maximum word length above which words will be ignored. Defaults to unbounded.

      • min_doc_freq number

        Ignore terms which do not occur in at least this many docs.

      • min_term_freq number

        Ignore words with less than this frequency in the source doc.

      • min_word_length number

        The minimum word length below which words will be ignored.

    • offsets boolean

      If true, the response includes term offsets.

    • payloads boolean

      If true, the response includes term payloads.

    • positions boolean

      If true, the response includes term positions.

    • routing string
    • term_statistics boolean

      If true, the response includes term frequency and document frequency.

    • version number
    • version_type string

      Values are internal, external, external_gte, or force.

  • ids array[string]

    A simplified syntax to specify documents by their ID if they're in the same index.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required
      Hide docs attributes Show docs attributes object
      • _id string
      • _index string Required
      • _version number
      • took number
      • found boolean
      • term_vectors object
        Hide term_vectors attribute Show term_vectors attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • field_statistics object
            Hide field_statistics attributes Show field_statistics attributes object
            • doc_count number Required
            • sum_doc_freq number Required
            • sum_ttf number Required
          • terms object Required
            Hide terms attribute Show terms attribute object
            • * object Additional properties
      • error object

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide error attributes Show error attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

GET /_mtermvectors
curl \
 --request GET 'http://api.example.com/_mtermvectors' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"docs\": [\n      {\n        \"_id\": \"2\",\n        \"fields\": [\n            \"message\"\n        ],\n        \"term_statistics\": true\n      },\n      {\n        \"_id\": \"1\"\n      }\n  ]\n}"'
Run `POST /my-index-000001/_mtermvectors`. When you specify an index in the request URI, the index does not need to be specified for each documents in the request body.
{
  "docs": [
      {
        "_id": "2",
        "fields": [
            "message"
        ],
        "term_statistics": true
      },
      {
        "_id": "1"
      }
  ]
}
Run `POST /my-index-000001/_mtermvectors`. If all requested documents are in same index and the parameters are the same, you can use a simplified syntax.
{
  "ids": [ "1", "2" ],
  "fields": [
    "message"
  ],
  "term_statistics": true
}
Run `POST /_mtermvectors` to generate term vectors for artificial documents provided in the body of the request. The mapping used is determined by the specified `_index`.
{
  "docs": [
      {
        "_index": "my-index-000001",
        "doc" : {
            "message" : "test test test"
        }
      },
      {
        "_index": "my-index-000001",
        "doc" : {
          "message" : "Another test ..."
        }
      }
  ]
}








Get multiple term vectors Generally available

POST /{index}/_mtermvectors

Get multiple term vectors with a single request. You can specify existing documents by index and ID or provide artificial documents in the body of the request. You can specify the index in the request body or request URI. The response contains a docs array with all the fetched termvectors. Each element has the structure provided by the termvectors API.

Artificial documents

You can also use mtermvectors to generate term vectors for artificial documents provided in the body of the request. The mapping used is determined by the specified _index.

Required authorization

  • Index privileges: read

Path parameters

  • index string Required

    The name of the index that contains the documents.

Query parameters

  • ids array[string]

    A comma-separated list of documents ids. You must define ids as parameter or set "ids" or "docs" in the request body

  • fields string | array[string]

    A comma-separated list or wildcard expressions of fields to include in the statistics. It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.

  • field_statistics boolean

    If true, the response includes the document count, sum of document frequencies, and sum of total term frequencies.

  • offsets boolean

    If true, the response includes term offsets.

  • payloads boolean

    If true, the response includes term payloads.

  • positions boolean

    If true, the response includes term positions.

  • preference string

    The node or shard the operation should be performed on. It is random by default.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • routing string

    A custom value used to route operations to a specific shard.

  • term_statistics boolean

    If true, the response includes term frequency and document frequency.

  • version number

    If true, returns the document version as part of a hit.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

application/json

Body

  • docs array[object]

    An array of existing or artificial documents.

    Hide docs attributes Show docs attributes object
    • _id string
    • _index string
    • doc object

      An artificial document (a document not present in the index) for which you want to retrieve term vectors.

    • fields string | array[string]
    • field_statistics boolean

      If true, the response includes the document count, sum of document frequencies, and sum of total term frequencies.

    • filter object
      Hide filter attributes Show filter attributes object
      • max_doc_freq number

        Ignore words which occur in more than this many docs. Defaults to unbounded.

      • max_num_terms number

        The maximum number of terms that must be returned per field.

      • max_term_freq number

        Ignore words with more than this frequency in the source doc. It defaults to unbounded.

      • max_word_length number

        The maximum word length above which words will be ignored. Defaults to unbounded.

      • min_doc_freq number

        Ignore terms which do not occur in at least this many docs.

      • min_term_freq number

        Ignore words with less than this frequency in the source doc.

      • min_word_length number

        The minimum word length below which words will be ignored.

    • offsets boolean

      If true, the response includes term offsets.

    • payloads boolean

      If true, the response includes term payloads.

    • positions boolean

      If true, the response includes term positions.

    • routing string
    • term_statistics boolean

      If true, the response includes term frequency and document frequency.

    • version number
    • version_type string

      Values are internal, external, external_gte, or force.

  • ids array[string]

    A simplified syntax to specify documents by their ID if they're in the same index.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required
      Hide docs attributes Show docs attributes object
      • _id string
      • _index string Required
      • _version number
      • took number
      • found boolean
      • term_vectors object
        Hide term_vectors attribute Show term_vectors attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • field_statistics object
            Hide field_statistics attributes Show field_statistics attributes object
            • doc_count number Required
            • sum_doc_freq number Required
            • sum_ttf number Required
          • terms object Required
            Hide terms attribute Show terms attribute object
            • * object Additional properties
      • error object

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide error attributes Show error attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

POST /{index}/_mtermvectors
curl \
 --request POST 'http://api.example.com/{index}/_mtermvectors' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"docs\": [\n      {\n        \"_id\": \"2\",\n        \"fields\": [\n            \"message\"\n        ],\n        \"term_statistics\": true\n      },\n      {\n        \"_id\": \"1\"\n      }\n  ]\n}"'
Run `POST /my-index-000001/_mtermvectors`. When you specify an index in the request URI, the index does not need to be specified for each documents in the request body.
{
  "docs": [
      {
        "_id": "2",
        "fields": [
            "message"
        ],
        "term_statistics": true
      },
      {
        "_id": "1"
      }
  ]
}
Run `POST /my-index-000001/_mtermvectors`. If all requested documents are in same index and the parameters are the same, you can use a simplified syntax.
{
  "ids": [ "1", "2" ],
  "fields": [
    "message"
  ],
  "term_statistics": true
}
Run `POST /_mtermvectors` to generate term vectors for artificial documents provided in the body of the request. The mapping used is determined by the specified `_index`.
{
  "docs": [
      {
        "_index": "my-index-000001",
        "doc" : {
            "message" : "test test test"
        }
      },
      {
        "_index": "my-index-000001",
        "doc" : {
          "message" : "Another test ..."
        }
      }
  ]
}








Get term vector information Generally available

GET /{index}/_termvectors/{id}

Get information and statistics about terms in the fields of a particular document.

You can retrieve term vectors for documents stored in the index or for artificial documents passed in the body of the request. You can specify the fields you are interested in through the fields parameter or by adding the fields to the request body. For example:

GET /my-index-000001/_termvectors/1?fields=message

Fields can be specified using wildcards, similar to the multi match query.

Term vectors are real-time by default, not near real-time. This can be changed by setting realtime parameter to false.

You can request three types of values: term information, term statistics, and field statistics. By default, all term information and field statistics are returned for all fields but term statistics are excluded.

Term information

  • term frequency in the field (always returned)
  • term positions (positions: true)
  • start and end offsets (offsets: true)
  • term payloads (payloads: true), as base64 encoded bytes

If the requested information wasn't stored in the index, it will be computed on the fly if possible. Additionally, term vectors could be computed for documents not even existing in the index, but instead provided by the user.


Start and end offsets assume UTF-16 encoding is being used. If you want to use these offsets in order to get the original text that produced this token, you should make sure that the string you are taking a sub-string of is also encoded using UTF-16.

Behaviour

The term and field statistics are not accurate. Deleted documents are not taken into account. The information is only retrieved for the shard the requested document resides in. The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context. By default, when requesting term vectors of artificial documents, a shard to get the statistics from is randomly selected. Use routing only to hit a particular shard.

Required authorization

  • Index privileges: read

Path parameters

  • index string Required

    The name of the index that contains the document.

  • id string Required

    A unique identifier for the document.

Query parameters

  • fields string | array[string]

    A comma-separated list or wildcard expressions of fields to include in the statistics. It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.

  • field_statistics boolean

    If true, the response includes:

    • The document count (how many documents contain this field).
    • The sum of document frequencies (the sum of document frequencies for all terms in this field).
    • The sum of total term frequencies (the sum of total term frequencies of each term in this field).
  • offsets boolean

    If true, the response includes term offsets.

  • payloads boolean

    If true, the response includes term payloads.

  • positions boolean

    If true, the response includes term positions.

  • preference string

    The node or shard the operation should be performed on. It is random by default.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • routing string

    A custom value that is used to route operations to a specific shard.

  • term_statistics boolean

    If true, the response includes:

    • The total term frequency (how often a term occurs in all documents).
    • The document frequency (the number of documents containing the current term).

    By default these values are not returned since term statistics can have a serious performance impact.

  • version number

    If true, returns the document version as part of a hit.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

application/json

Body

  • doc object

    An artificial document (a document not present in the index) for which you want to retrieve term vectors.

  • filter object
    Hide filter attributes Show filter attributes object
    • max_doc_freq number

      Ignore words which occur in more than this many docs. Defaults to unbounded.

    • max_num_terms number

      The maximum number of terms that must be returned per field.

    • max_term_freq number

      Ignore words with more than this frequency in the source doc. It defaults to unbounded.

    • max_word_length number

      The maximum word length above which words will be ignored. Defaults to unbounded.

    • min_doc_freq number

      Ignore terms which do not occur in at least this many docs.

    • min_term_freq number

      Ignore words with less than this frequency in the source doc.

    • min_word_length number

      The minimum word length below which words will be ignored.

  • per_field_analyzer object

    Override the default per-field analyzer. This is useful in order to generate term vectors in any fashion, especially when using artificial documents. When providing an analyzer for a field that already stores term vectors, the term vectors will be regenerated.

    Hide per_field_analyzer attribute Show per_field_analyzer attribute object
    • * string Additional properties
  • fields string | array[string]
  • field_statistics boolean

    If true, the response includes:

    • The document count (how many documents contain this field).
    • The sum of document frequencies (the sum of document frequencies for all terms in this field).
    • The sum of total term frequencies (the sum of total term frequencies of each term in this field).
  • offsets boolean

    If true, the response includes term offsets.

  • payloads boolean

    If true, the response includes term payloads.

  • positions boolean

    If true, the response includes term positions.

  • term_statistics boolean

    If true, the response includes:

    • The total term frequency (how often a term occurs in all documents).
    • The document frequency (the number of documents containing the current term).

    By default these values are not returned since term statistics can have a serious performance impact.

  • routing string
  • version number
  • version_type string

    Values are internal, external, external_gte, or force.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • found boolean Required
    • _id string
    • _index string Required
    • term_vectors object
      Hide term_vectors attribute Show term_vectors attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • field_statistics object
          Hide field_statistics attributes Show field_statistics attributes object
          • doc_count number Required
          • sum_doc_freq number Required
          • sum_ttf number Required
        • terms object Required
          Hide terms attribute Show terms attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • doc_freq number
            • score number
            • term_freq number Required
            • tokens array[object]
            • ttf number
    • took number Required
    • _version number Required
GET /{index}/_termvectors/{id}
curl \
 --request GET 'http://api.example.com/{index}/_termvectors/{id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"fields\" : [\"text\"],\n  \"offsets\" : true,\n  \"payloads\" : true,\n  \"positions\" : true,\n  \"term_statistics\" : true,\n  \"field_statistics\" : true\n}"'
Run `GET /my-index-000001/_termvectors/1` to return all information and statistics for field `text` in document 1.
{
  "fields" : ["text"],
  "offsets" : true,
  "payloads" : true,
  "positions" : true,
  "term_statistics" : true,
  "field_statistics" : true
}
Run `GET /my-index-000001/_termvectors/1` to set per-field analyzers. A different analyzer than the one at the field may be provided by using the `per_field_analyzer` parameter.
{
  "doc" : {
    "fullname" : "John Doe",
    "text" : "test test test"
  },
  "fields": ["fullname"],
  "per_field_analyzer" : {
    "fullname": "keyword"
  }
}
Run `GET /imdb/_termvectors` to filter the terms returned based on their tf-idf scores. It returns the three most "interesting" keywords from the artificial document having the given "plot" field value. Notice that the keyword "Tony" or any stop words are not part of the response, as their tf-idf must be too low.
{
  "doc": {
    "plot": "When wealthy industrialist Tony Stark is forced to build an armored suit after a life-threatening incident, he ultimately decides to use its technology to fight against evil."
  },
  "term_statistics": true,
  "field_statistics": true,
  "positions": false,
  "offsets": false,
  "filter": {
    "max_num_terms": 3,
    "min_term_freq": 1,
    "min_doc_freq": 1
  }
}
Run `GET /my-index-000001/_termvectors/1`. Term vectors which are not explicitly stored in the index are automatically computed on the fly. This request returns all information and statistics for the fields in document 1, even though the terms haven't been explicitly stored in the index. Note that for the field text, the terms are not regenerated.
{
  "fields" : ["text", "some_field_without_term_vectors"],
  "offsets" : true,
  "positions" : true,
  "term_statistics" : true,
  "field_statistics" : true
}
Run `GET /my-index-000001/_termvectors`. Term vectors can be generated for artificial documents, that is for documents not present in the index. If dynamic mapping is turned on (default), the document fields not in the original mapping will be dynamically created.
{
  "doc" : {
    "fullname" : "John Doe",
    "text" : "test test test"
  }
}
Response examples (200)
A successful response from `GET /my-index-000001/_termvectors/1`.
{
  "_index": "my-index-000001",
  "_id": "1",
  "_version": 1,
  "found": true,
  "took": 6,
  "term_vectors": {
    "text": {
      "field_statistics": {
        "sum_doc_freq": 4,
        "doc_count": 2,
        "sum_ttf": 6
      },
      "terms": {
        "test": {
          "doc_freq": 2,
          "ttf": 4,
          "term_freq": 3,
          "tokens": [
            {
              "position": 0,
              "start_offset": 0,
              "end_offset": 4,
              "payload": "d29yZA=="
            },
            {
              "position": 1,
              "start_offset": 5,
              "end_offset": 9,
              "payload": "d29yZA=="
            },
            {
              "position": 2,
              "start_offset": 10,
              "end_offset": 14,
              "payload": "d29yZA=="
            }
          ]
        }
      }
    }
  }
}
A successful response from `GET /my-index-000001/_termvectors` with `per_field_analyzer` in the request body.
{
  "_index": "my-index-000001",
  "_version": 0,
  "found": true,
  "took": 6,
  "term_vectors": {
    "fullname": {
      "field_statistics": {
          "sum_doc_freq": 2,
          "doc_count": 4,
          "sum_ttf": 4
      },
      "terms": {
          "John Doe": {
            "term_freq": 1,
            "tokens": [
                {
                  "position": 0,
                  "start_offset": 0,
                  "end_offset": 8
                }
            ]
          }
      }
    }
  }
}
A successful response from `GET /my-index-000001/_termvectors` with a `filter` in the request body.
{
  "_index": "imdb",
  "_version": 0,
  "found": true,
  "term_vectors": {
      "plot": {
        "field_statistics": {
            "sum_doc_freq": 3384269,
            "doc_count": 176214,
            "sum_ttf": 3753460
        },
        "terms": {
            "armored": {
              "doc_freq": 27,
              "ttf": 27,
              "term_freq": 1,
              "score": 9.74725
            },
            "industrialist": {
              "doc_freq": 88,
              "ttf": 88,
              "term_freq": 1,
              "score": 8.590818
            },
            "stark": {
              "doc_freq": 44,
              "ttf": 47,
              "term_freq": 1,
              "score": 9.272792
            }
        }
      }
  }
}

Get term vector information Generally available

POST /{index}/_termvectors/{id}

Get information and statistics about terms in the fields of a particular document.

You can retrieve term vectors for documents stored in the index or for artificial documents passed in the body of the request. You can specify the fields you are interested in through the fields parameter or by adding the fields to the request body. For example:

GET /my-index-000001/_termvectors/1?fields=message

Fields can be specified using wildcards, similar to the multi match query.

Term vectors are real-time by default, not near real-time. This can be changed by setting realtime parameter to false.

You can request three types of values: term information, term statistics, and field statistics. By default, all term information and field statistics are returned for all fields but term statistics are excluded.

Term information

  • term frequency in the field (always returned)
  • term positions (positions: true)
  • start and end offsets (offsets: true)
  • term payloads (payloads: true), as base64 encoded bytes

If the requested information wasn't stored in the index, it will be computed on the fly if possible. Additionally, term vectors could be computed for documents not even existing in the index, but instead provided by the user.


Start and end offsets assume UTF-16 encoding is being used. If you want to use these offsets in order to get the original text that produced this token, you should make sure that the string you are taking a sub-string of is also encoded using UTF-16.

Behaviour

The term and field statistics are not accurate. Deleted documents are not taken into account. The information is only retrieved for the shard the requested document resides in. The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context. By default, when requesting term vectors of artificial documents, a shard to get the statistics from is randomly selected. Use routing only to hit a particular shard.

Required authorization

  • Index privileges: read

Path parameters

  • index string Required

    The name of the index that contains the document.

  • id string Required

    A unique identifier for the document.

Query parameters

  • fields string | array[string]

    A comma-separated list or wildcard expressions of fields to include in the statistics. It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.

  • field_statistics boolean

    If true, the response includes:

    • The document count (how many documents contain this field).
    • The sum of document frequencies (the sum of document frequencies for all terms in this field).
    • The sum of total term frequencies (the sum of total term frequencies of each term in this field).
  • offsets boolean

    If true, the response includes term offsets.

  • payloads boolean

    If true, the response includes term payloads.

  • positions boolean

    If true, the response includes term positions.

  • preference string

    The node or shard the operation should be performed on. It is random by default.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • routing string

    A custom value that is used to route operations to a specific shard.

  • term_statistics boolean

    If true, the response includes:

    • The total term frequency (how often a term occurs in all documents).
    • The document frequency (the number of documents containing the current term).

    By default these values are not returned since term statistics can have a serious performance impact.

  • version number

    If true, returns the document version as part of a hit.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

application/json

Body

  • doc object

    An artificial document (a document not present in the index) for which you want to retrieve term vectors.

  • filter object
    Hide filter attributes Show filter attributes object
    • max_doc_freq number

      Ignore words which occur in more than this many docs. Defaults to unbounded.

    • max_num_terms number

      The maximum number of terms that must be returned per field.

    • max_term_freq number

      Ignore words with more than this frequency in the source doc. It defaults to unbounded.

    • max_word_length number

      The maximum word length above which words will be ignored. Defaults to unbounded.

    • min_doc_freq number

      Ignore terms which do not occur in at least this many docs.

    • min_term_freq number

      Ignore words with less than this frequency in the source doc.

    • min_word_length number

      The minimum word length below which words will be ignored.

  • per_field_analyzer object

    Override the default per-field analyzer. This is useful in order to generate term vectors in any fashion, especially when using artificial documents. When providing an analyzer for a field that already stores term vectors, the term vectors will be regenerated.

    Hide per_field_analyzer attribute Show per_field_analyzer attribute object
    • * string Additional properties
  • fields string | array[string]
  • field_statistics boolean

    If true, the response includes:

    • The document count (how many documents contain this field).
    • The sum of document frequencies (the sum of document frequencies for all terms in this field).
    • The sum of total term frequencies (the sum of total term frequencies of each term in this field).
  • offsets boolean

    If true, the response includes term offsets.

  • payloads boolean

    If true, the response includes term payloads.

  • positions boolean

    If true, the response includes term positions.

  • term_statistics boolean

    If true, the response includes:

    • The total term frequency (how often a term occurs in all documents).
    • The document frequency (the number of documents containing the current term).

    By default these values are not returned since term statistics can have a serious performance impact.

  • routing string
  • version number
  • version_type string

    Values are internal, external, external_gte, or force.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • found boolean Required
    • _id string
    • _index string Required
    • term_vectors object
      Hide term_vectors attribute Show term_vectors attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • field_statistics object
          Hide field_statistics attributes Show field_statistics attributes object
          • doc_count number Required
          • sum_doc_freq number Required
          • sum_ttf number Required
        • terms object Required
          Hide terms attribute Show terms attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • doc_freq number
            • score number
            • term_freq number Required
            • tokens array[object]
            • ttf number
    • took number Required
    • _version number Required
POST /{index}/_termvectors/{id}
curl \
 --request POST 'http://api.example.com/{index}/_termvectors/{id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"fields\" : [\"text\"],\n  \"offsets\" : true,\n  \"payloads\" : true,\n  \"positions\" : true,\n  \"term_statistics\" : true,\n  \"field_statistics\" : true\n}"'
Run `GET /my-index-000001/_termvectors/1` to return all information and statistics for field `text` in document 1.
{
  "fields" : ["text"],
  "offsets" : true,
  "payloads" : true,
  "positions" : true,
  "term_statistics" : true,
  "field_statistics" : true
}
Run `GET /my-index-000001/_termvectors/1` to set per-field analyzers. A different analyzer than the one at the field may be provided by using the `per_field_analyzer` parameter.
{
  "doc" : {
    "fullname" : "John Doe",
    "text" : "test test test"
  },
  "fields": ["fullname"],
  "per_field_analyzer" : {
    "fullname": "keyword"
  }
}
Run `GET /imdb/_termvectors` to filter the terms returned based on their tf-idf scores. It returns the three most "interesting" keywords from the artificial document having the given "plot" field value. Notice that the keyword "Tony" or any stop words are not part of the response, as their tf-idf must be too low.
{
  "doc": {
    "plot": "When wealthy industrialist Tony Stark is forced to build an armored suit after a life-threatening incident, he ultimately decides to use its technology to fight against evil."
  },
  "term_statistics": true,
  "field_statistics": true,
  "positions": false,
  "offsets": false,
  "filter": {
    "max_num_terms": 3,
    "min_term_freq": 1,
    "min_doc_freq": 1
  }
}
Run `GET /my-index-000001/_termvectors/1`. Term vectors which are not explicitly stored in the index are automatically computed on the fly. This request returns all information and statistics for the fields in document 1, even though the terms haven't been explicitly stored in the index. Note that for the field text, the terms are not regenerated.
{
  "fields" : ["text", "some_field_without_term_vectors"],
  "offsets" : true,
  "positions" : true,
  "term_statistics" : true,
  "field_statistics" : true
}
Run `GET /my-index-000001/_termvectors`. Term vectors can be generated for artificial documents, that is for documents not present in the index. If dynamic mapping is turned on (default), the document fields not in the original mapping will be dynamically created.
{
  "doc" : {
    "fullname" : "John Doe",
    "text" : "test test test"
  }
}
Response examples (200)
A successful response from `GET /my-index-000001/_termvectors/1`.
{
  "_index": "my-index-000001",
  "_id": "1",
  "_version": 1,
  "found": true,
  "took": 6,
  "term_vectors": {
    "text": {
      "field_statistics": {
        "sum_doc_freq": 4,
        "doc_count": 2,
        "sum_ttf": 6
      },
      "terms": {
        "test": {
          "doc_freq": 2,
          "ttf": 4,
          "term_freq": 3,
          "tokens": [
            {
              "position": 0,
              "start_offset": 0,
              "end_offset": 4,
              "payload": "d29yZA=="
            },
            {
              "position": 1,
              "start_offset": 5,
              "end_offset": 9,
              "payload": "d29yZA=="
            },
            {
              "position": 2,
              "start_offset": 10,
              "end_offset": 14,
              "payload": "d29yZA=="
            }
          ]
        }
      }
    }
  }
}
A successful response from `GET /my-index-000001/_termvectors` with `per_field_analyzer` in the request body.
{
  "_index": "my-index-000001",
  "_version": 0,
  "found": true,
  "took": 6,
  "term_vectors": {
    "fullname": {
      "field_statistics": {
          "sum_doc_freq": 2,
          "doc_count": 4,
          "sum_ttf": 4
      },
      "terms": {
          "John Doe": {
            "term_freq": 1,
            "tokens": [
                {
                  "position": 0,
                  "start_offset": 0,
                  "end_offset": 8
                }
            ]
          }
      }
    }
  }
}
A successful response from `GET /my-index-000001/_termvectors` with a `filter` in the request body.
{
  "_index": "imdb",
  "_version": 0,
  "found": true,
  "term_vectors": {
      "plot": {
        "field_statistics": {
            "sum_doc_freq": 3384269,
            "doc_count": 176214,
            "sum_ttf": 3753460
        },
        "terms": {
            "armored": {
              "doc_freq": 27,
              "ttf": 27,
              "term_freq": 1,
              "score": 9.74725
            },
            "industrialist": {
              "doc_freq": 88,
              "ttf": 88,
              "term_freq": 1,
              "score": 8.590818
            },
            "stark": {
              "doc_freq": 44,
              "ttf": 47,
              "term_freq": 1,
              "score": 9.272792
            }
        }
      }
  }
}

Get term vector information Generally available

GET /{index}/_termvectors

Get information and statistics about terms in the fields of a particular document.

You can retrieve term vectors for documents stored in the index or for artificial documents passed in the body of the request. You can specify the fields you are interested in through the fields parameter or by adding the fields to the request body. For example:

GET /my-index-000001/_termvectors/1?fields=message

Fields can be specified using wildcards, similar to the multi match query.

Term vectors are real-time by default, not near real-time. This can be changed by setting realtime parameter to false.

You can request three types of values: term information, term statistics, and field statistics. By default, all term information and field statistics are returned for all fields but term statistics are excluded.

Term information

  • term frequency in the field (always returned)
  • term positions (positions: true)
  • start and end offsets (offsets: true)
  • term payloads (payloads: true), as base64 encoded bytes

If the requested information wasn't stored in the index, it will be computed on the fly if possible. Additionally, term vectors could be computed for documents not even existing in the index, but instead provided by the user.


Start and end offsets assume UTF-16 encoding is being used. If you want to use these offsets in order to get the original text that produced this token, you should make sure that the string you are taking a sub-string of is also encoded using UTF-16.

Behaviour

The term and field statistics are not accurate. Deleted documents are not taken into account. The information is only retrieved for the shard the requested document resides in. The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context. By default, when requesting term vectors of artificial documents, a shard to get the statistics from is randomly selected. Use routing only to hit a particular shard.

Required authorization

  • Index privileges: read

Path parameters

  • index string Required

    The name of the index that contains the document.

Query parameters

  • fields string | array[string]

    A comma-separated list or wildcard expressions of fields to include in the statistics. It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.

  • field_statistics boolean

    If true, the response includes:

    • The document count (how many documents contain this field).
    • The sum of document frequencies (the sum of document frequencies for all terms in this field).
    • The sum of total term frequencies (the sum of total term frequencies of each term in this field).
  • offsets boolean

    If true, the response includes term offsets.

  • payloads boolean

    If true, the response includes term payloads.

  • positions boolean

    If true, the response includes term positions.

  • preference string

    The node or shard the operation should be performed on. It is random by default.

  • realtime boolean

    If true, the request is real-time as opposed to near-real-time.

  • routing string

    A custom value that is used to route operations to a specific shard.

  • term_statistics boolean

    If true, the response includes:

    • The total term frequency (how often a term occurs in all documents).
    • The document frequency (the number of documents containing the current term).

    By default these values are not returned since term statistics can have a serious performance impact.

  • version number

    If true, returns the document version as part of a hit.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

application/json

Body

  • doc object

    An artificial document (a document not present in the index) for which you want to retrieve term vectors.

  • filter object
    Hide filter attributes Show filter attributes object
    • max_doc_freq number

      Ignore words which occur in more than this many docs. Defaults to unbounded.

    • max_num_terms number

      The maximum number of terms that must be returned per field.

    • max_term_freq number

      Ignore words with more than this frequency in the source doc. It defaults to unbounded.

    • max_word_length number

      The maximum word length above which words will be ignored. Defaults to unbounded.

    • min_doc_freq number

      Ignore terms which do not occur in at least this many docs.

    • min_term_freq number

      Ignore words with less than this frequency in the source doc.

    • min_word_length number

      The minimum word length below which words will be ignored.

  • per_field_analyzer object

    Override the default per-field analyzer. This is useful in order to generate term vectors in any fashion, especially when using artificial documents. When providing an analyzer for a field that already stores term vectors, the term vectors will be regenerated.

    Hide per_field_analyzer attribute Show per_field_analyzer attribute object
    • * string Additional properties
  • fields string | array[string]
  • field_statistics boolean

    If true, the response includes:

    • The document count (how many documents contain this field).
    • The sum of document frequencies (the sum of document frequencies for all terms in this field).
    • The sum of total term frequencies (the sum of total term frequencies of each term in this field).
  • offsets boolean

    If true, the response includes term offsets.

  • payloads boolean

    If true, the response includes term payloads.

  • positions boolean

    If true, the response includes term positions.

  • term_statistics boolean

    If true, the response includes:

    • The total term frequency (how often a term occurs in all documents).
    • The document frequency (the number of documents containing the current term).

    By default these values are not returned since term statistics can have a serious performance impact.

  • routing string
  • version number
  • version_type string

    Values are internal, external, external_gte, or force.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • found boolean Required
    • _id string
    • _index string Required
    • term_vectors object
      Hide term_vectors attribute Show term_vectors attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • field_statistics object
          Hide field_statistics attributes Show field_statistics attributes object
          • doc_count number Required
          • sum_doc_freq number Required
          • sum_ttf number Required
        • terms object Required
          Hide terms attribute Show terms attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • doc_freq number
            • score number
            • term_freq number Required
            • tokens array[object]
            • ttf number
    • took number Required
    • _version number Required
GET /{index}/_termvectors
curl \
 --request GET 'http://api.example.com/{index}/_termvectors' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"fields\" : [\"text\"],\n  \"offsets\" : true,\n  \"payloads\" : true,\n  \"positions\" : true,\n  \"term_statistics\" : true,\n  \"field_statistics\" : true\n}"'
Run `GET /my-index-000001/_termvectors/1` to return all information and statistics for field `text` in document 1.
{
  "fields" : ["text"],
  "offsets" : true,
  "payloads" : true,
  "positions" : true,
  "term_statistics" : true,
  "field_statistics" : true
}
Run `GET /my-index-000001/_termvectors/1` to set per-field analyzers. A different analyzer than the one at the field may be provided by using the `per_field_analyzer` parameter.
{
  "doc" : {
    "fullname" : "John Doe",
    "text" : "test test test"
  },
  "fields": ["fullname"],
  "per_field_analyzer" : {
    "fullname": "keyword"
  }
}
Run `GET /imdb/_termvectors` to filter the terms returned based on their tf-idf scores. It returns the three most "interesting" keywords from the artificial document having the given "plot" field value. Notice that the keyword "Tony" or any stop words are not part of the response, as their tf-idf must be too low.
{
  "doc": {
    "plot": "When wealthy industrialist Tony Stark is forced to build an armored suit after a life-threatening incident, he ultimately decides to use its technology to fight against evil."
  },
  "term_statistics": true,
  "field_statistics": true,
  "positions": false,
  "offsets": false,
  "filter": {
    "max_num_terms": 3,
    "min_term_freq": 1,
    "min_doc_freq": 1
  }
}
Run `GET /my-index-000001/_termvectors/1`. Term vectors which are not explicitly stored in the index are automatically computed on the fly. This request returns all information and statistics for the fields in document 1, even though the terms haven't been explicitly stored in the index. Note that for the field text, the terms are not regenerated.
{
  "fields" : ["text", "some_field_without_term_vectors"],
  "offsets" : true,
  "positions" : true,
  "term_statistics" : true,
  "field_statistics" : true
}
Run `GET /my-index-000001/_termvectors`. Term vectors can be generated for artificial documents, that is for documents not present in the index. If dynamic mapping is turned on (default), the document fields not in the original mapping will be dynamically created.
{
  "doc" : {
    "fullname" : "John Doe",
    "text" : "test test test"
  }
}
Response examples (200)
A successful response from `GET /my-index-000001/_termvectors/1`.
{
  "_index": "my-index-000001",
  "_id": "1",
  "_version": 1,
  "found": true,
  "took": 6,
  "term_vectors": {
    "text": {
      "field_statistics": {
        "sum_doc_freq": 4,
        "doc_count": 2,
        "sum_ttf": 6
      },
      "terms": {
        "test": {
          "doc_freq": 2,
          "ttf": 4,
          "term_freq": 3,
          "tokens": [
            {
              "position": 0,
              "start_offset": 0,
              "end_offset": 4,
              "payload": "d29yZA=="
            },
            {
              "position": 1,
              "start_offset": 5,
              "end_offset": 9,
              "payload": "d29yZA=="
            },
            {
              "position": 2,
              "start_offset": 10,
              "end_offset": 14,
              "payload": "d29yZA=="
            }
          ]
        }
      }
    }
  }
}
A successful response from `GET /my-index-000001/_termvectors` with `per_field_analyzer` in the request body.
{
  "_index": "my-index-000001",
  "_version": 0,
  "found": true,
  "took": 6,
  "term_vectors": {
    "fullname": {
      "field_statistics": {
          "sum_doc_freq": 2,
          "doc_count": 4,
          "sum_ttf": 4
      },
      "terms": {
          "John Doe": {
            "term_freq": 1,
            "tokens": [
                {
                  "position": 0,
                  "start_offset": 0,
                  "end_offset": 8
                }
            ]
          }
      }
    }
  }
}
A successful response from `GET /my-index-000001/_termvectors` with a `filter` in the request body.
{
  "_index": "imdb",
  "_version": 0,
  "found": true,
  "term_vectors": {
      "plot": {
        "field_statistics": {
            "sum_doc_freq": 3384269,
            "doc_count": 176214,
            "sum_ttf": 3753460
        },
        "terms": {
            "armored": {
              "doc_freq": 27,
              "ttf": 27,
              "term_freq": 1,
              "score": 9.74725
            },
            "industrialist": {
              "doc_freq": 88,
              "ttf": 88,
              "term_freq": 1,
              "score": 8.590818
            },
            "stark": {
              "doc_freq": 44,
              "ttf": 47,
              "term_freq": 1,
              "score": 9.272792
            }
        }
      }
  }
}
















Enrich

Get an enrich policy Generally available; Added in 7.5.0

GET /_enrich/policy/{name}

Returns information about an enrich policy.

Path parameters

  • name string | array[string] Required

    Comma-separated list of enrich policy names used to limit the request. To return information for all enrich policies, omit this parameter.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • policies array[object] Required
      Hide policies attribute Show policies attribute object
      • config object Required
        Hide config attribute Show config attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • enrich_fields string | array[string] Required
          • indices string | array[string] Required
          • match_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • query object

            An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

            External documentation
          • name string
          • elasticsearch_version string
GET /_enrich/policy/{name}
curl \
 --request GET 'http://api.example.com/_enrich/policy/{name}' \
 --header "Authorization: $API_KEY"

Create an enrich policy Generally available; Added in 7.5.0

PUT /_enrich/policy/{name}

Creates an enrich policy.

Path parameters

  • name string Required

    Name of the enrich policy to create or update.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

application/json

Body Required

  • geo_match object Additional properties
    Hide geo_match attributes Show geo_match attributes object
    • enrich_fields string | array[string] Required
    • indices string | array[string] Required
    • match_field string Required

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • query object

      An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

      External documentation
    • name string
    • elasticsearch_version string
  • match object Additional properties
    Hide match attributes Show match attributes object
    • enrich_fields string | array[string] Required
    • indices string | array[string] Required
    • match_field string Required

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • query object

      An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

      External documentation
    • name string
    • elasticsearch_version string
  • range object Additional properties
    Hide range attributes Show range attributes object
    • enrich_fields string | array[string] Required
    • indices string | array[string] Required
    • match_field string Required

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • query object

      An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

      External documentation
    • name string
    • elasticsearch_version string

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_enrich/policy/{name}
curl \
 --request PUT 'http://api.example.com/_enrich/policy/{name}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"geo_match\": {\n    \"indices\": \"postal_codes\",\n    \"match_field\": \"location\",\n    \"enrich_fields\": [ \"location\", \"postal_code\" ]\n  }\n}"'
Request example
An example body for a `PUT /_enrich/policy/postal_policy` request.
{
  "geo_match": {
    "indices": "postal_codes",
    "match_field": "location",
    "enrich_fields": [ "location", "postal_code" ]
  }
}








Get an enrich policy Generally available; Added in 7.5.0

GET /_enrich/policy

Returns information about an enrich policy.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • policies array[object] Required
      Hide policies attribute Show policies attribute object
      • config object Required
        Hide config attribute Show config attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • enrich_fields string | array[string] Required
          • indices string | array[string] Required
          • match_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • query object

            An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

            External documentation
          • name string
          • elasticsearch_version string
GET /_enrich/policy
curl \
 --request GET 'http://api.example.com/_enrich/policy' \
 --header "Authorization: $API_KEY"

Get enrich stats Generally available; Added in 7.5.0

GET /_enrich/_stats

Returns enrich coordinator statistics and information about enrich policies that are currently executing.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • coordinator_stats array[object] Required

      Objects containing information about each coordinating ingest node for configured enrich processors.

      Hide coordinator_stats attributes Show coordinator_stats attributes object
      • executed_searches_total number Required
      • node_id string Required
      • queue_size number Required
      • remote_requests_current number Required
      • remote_requests_total number Required
    • executing_policies array[object] Required

      Objects containing information about each enrich policy that is currently executing.

      Hide executing_policies attributes Show executing_policies attributes object
      • name string Required
      • task object Required Additional properties
        Hide task attributes Show task attributes object
        • action string Required
        • cancelled boolean
        • cancellable boolean Required
        • description string

          Human readable text that identifies the particular request that the task is performing. For example, it might identify the search request being performed by a search task. Other kinds of tasks have different descriptions, like _reindex which has the source and the destination, or _bulk which just has the number of requests and the destination indices. Many requests will have only an empty description because more detailed information about the request is not easily available or particularly helpful in identifying the request.

        • headers object Required
          Hide headers attribute Show headers attribute object
          • * string Additional properties
        • id number Required
        • node string Required
        • running_time string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • running_time_in_nanos number

          Time unit for nanoseconds

        • start_time_in_millis number

          Time unit for milliseconds

        • status object

          The internal status of the task, which varies from task to task. The format also varies. While the goal is to keep the status for a particular task consistent from version to version, this is not always possible because sometimes the implementation changes. Fields might be removed from the status for a particular request so any parsing you do of the status might break in minor releases.

        • type string Required
        • parent_task_id string | number

    • cache_stats array[object] Generally available; Added in 7.16.0

      Objects containing information about the enrich cache stats on each ingest node.

      Hide cache_stats attributes Show cache_stats attributes object
      • node_id string Required
      • count number Required
      • hits number Required
      • hits_time_in_millis number

        Time unit for milliseconds

      • misses number Required
      • misses_time_in_millis number

        Time unit for milliseconds

      • evictions number Required
      • size_in_bytes number Required
GET /_enrich/_stats
curl \
 --request GET 'http://api.example.com/_enrich/_stats' \
 --header "Authorization: $API_KEY"









Get the async EQL status Generally available; Added in 7.9.0

GET /_eql/search/status/{id}

Get the current status for an async EQL search or a stored synchronous EQL search without returning results.

Path parameters

  • id string Required

    Identifier for the search.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string Required
    • is_partial boolean Required

      If true, the search request is still executing. If false, the search is completed.

    • is_running boolean Required

      If true, the response does not contain complete search results. This could be because either the search is still running (is_running status is false), or because it is already completed (is_running status is true) and results are partial due to failures or timeouts.

    • start_time_in_millis number

      Time unit for milliseconds

    • expiration_time_in_millis number

      Time unit for milliseconds

    • completion_status number

      For a completed search shows the http status code of the completed search.

GET /_eql/search/status/{id}
curl \
 --request GET 'http://api.example.com/_eql/search/status/{id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response for getting status information for an async EQL search.
{
  "id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  "is_running" : true,
  "is_partial" : true,
  "start_time_in_millis" : 1611690235000,
  "expiration_time_in_millis" : 1611690295000
}




Get EQL search results Generally available; Added in 7.9.0

POST /{index}/_eql/search

Returns search results for an Event Query Language (EQL) query. EQL assumes each document in a data stream or index corresponds to an event.

External documentation

Path parameters

  • index string | array[string] Required

    The name of the index to scope the operation

Query parameters

  • allow_no_indices boolean
  • allow_partial_search_results boolean

    If true, returns partial results if there are shard failures. If false, returns an error with no partial results.

  • allow_partial_sequence_results boolean

    If true, sequence queries will return partial results in case of shard failures. If false, they will return no results at all. This flag has effect only if allow_partial_search_results is true.

  • expand_wildcards string | array[string]

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If true, missing or closed indices are not included in the response.

  • keep_alive string

    Period for which the search and its results are stored on the cluster.

    Values are -1 or 0.

  • keep_on_completion boolean

    If true, the search and its results are stored on the cluster.

  • wait_for_completion_timeout string

    Timeout duration to wait for the request to finish. Defaults to no timeout, meaning the request waits for complete search results.

    Values are -1 or 0.

application/json

Body Required

  • query string Required

    EQL query you wish to run.

  • case_sensitive boolean
  • event_category_field string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • tiebreaker_field string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • timestamp_field string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • fetch_size number
  • filter object | array[object]

    Query, written in Query DSL, used to filter the events on which the EQL query runs.

    One of:

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation
  • keep_alive string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • keep_on_completion boolean
  • wait_for_completion_timeout string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • allow_partial_search_results boolean

    Allow query execution also in case of shard failures. If true, the query will keep running and will return results based on the available shards. For sequences, the behavior can be further refined using allow_partial_sequence_results

  • allow_partial_sequence_results boolean

    This flag applies only to sequences and has effect only if allow_partial_search_results=true. If true, the sequence query will return results based on the available shards, ignoring the others. If false, the sequence query will return successfully, but will always have empty results.

  • size number
  • fields object | array[object]

    Array of wildcard (*) patterns. The response returns values for field names matching these patterns in the fields property of each hit.

    One of:

    A reference to a field with formatting instructions on how to return the value

    Hide attributes Show attributes
    • field string Required

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • format string

      The format in which the values are returned.

    • include_unmapped boolean
  • result_position string

    Values are tail or head.

  • runtime_mappings object
    Hide runtime_mappings attribute Show runtime_mappings attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • fields object

        For type composite

        Hide fields attribute Show fields attribute object
        • * object Additional properties
          Hide * attribute Show * attribute object
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • fetch_fields array[object]

        For type lookup

        Hide fetch_fields attributes Show fetch_fields attributes object
        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • format string
      • format string

        A custom format for date type runtime fields.

      • input_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_index string
      • script object
        Hide script attributes Show script attributes object
        • source string

          The script source.

        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          Hide params attribute Show params attribute object
          • * object Additional properties
        • lang string

          Any of:

          Values are painless, expression, mustache, or java.

        • options object
          Hide options attribute Show options attribute object
          • * string Additional properties
      • type string Required

        Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

  • max_samples_per_key number

    By default, the response of a sample query contains up to 10 samples, with one sample per unique set of join keys. Use the size parameter to get a smaller or larger set of samples. To retrieve more than one sample per set of join keys, use the max_samples_per_key parameter. Pipes are not supported for sample queries.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string
    • is_partial boolean

      If true, the response does not contain complete search results.

    • is_running boolean

      If true, the search request is still executing.

    • took number

      Time unit for milliseconds

    • timed_out boolean

      If true, the request timed out before completion.

    • hits object Required
      Hide hits attributes Show hits attributes object
      • total object
        Hide total attributes Show total attributes object
        • relation string Required

          Values are eq or gte.

        • value number Required
      • events array[object]

        Contains events matching the query. Each object represents a matching event.

        Hide events attributes Show events attributes object
        • _index string Required
        • _id string Required
        • _source object Required

          Original JSON body passed for the event at index time.

        • missing boolean

          Set to true for events in a timespan-constrained sequence that do not meet a given condition.

        • fields object
          Hide fields attribute Show fields attribute object
          • * array[object] Additional properties
      • sequences array[object]

        Contains event sequences matching the query. Each object represents a matching sequence. This parameter is only returned for EQL queries containing a sequence.

        Hide sequences attributes Show sequences attributes object
        • events array[object] Required

          Contains events matching the query. Each object represents a matching event.

          Hide events attributes Show events attributes object
          • _index string Required
          • _id string Required
          • _source object Required

            Original JSON body passed for the event at index time.

          • missing boolean

            Set to true for events in a timespan-constrained sequence that do not meet a given condition.

          • fields object
        • join_keys array[object]

          Shared field values used to constrain matches in the sequence. These are defined using the by keyword in the EQL query syntax.

    • shard_failures array[object]

      Contains information about shard failures (if any), in case allow_partial_search_results=true

      Hide shard_failures attributes Show shard_failures attributes object
      • index string
      • node string
      • reason object Required

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide reason attributes Show reason attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • shard number Required
      • status string
POST /{index}/_eql/search
curl \
 --request POST 'http://api.example.com/{index}/_eql/search' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"query\": \"\"\"\n    process where (process.name == \"cmd.exe\" and process.pid != 2013)\n  \"\"\"\n}"'
Request examples
Run `GET /my-data-stream/_eql/search` to search for events that have a `process.name` of `cmd.exe` and a `process.pid` other than `2013`.
{
  "query": """
    process where (process.name == "cmd.exe" and process.pid != 2013)
  """
}
Run `GET /my-data-stream/_eql/search` to search for a sequence of events. The sequence starts with an event with an `event.category` of `file`, a `file.name` of `cmd.exe`, and a `process.pid` other than `2013`. It is followed by an event with an `event.category` of `process` and a `process.executable` that contains the substring `regsvr32`. These events must also share the same `process.pid` value.
{
  "query": """
    sequence by process.pid
      [ file where file.name == "cmd.exe" and process.pid != 2013 ]
      [ process where stringContains(process.executable, "regsvr32") ]
  """
}
Response examples (200)
{
  "is_partial": false,
  "is_running": false,
  "took": 6,
  "timed_out": false,
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "sequences": [
      {
        "join_keys": [
          2012
        ],
        "events": [
          {
            "_index": ".ds-my-data-stream-2099.12.07-000001",
            "_id": "AtOJ4UjUBAAx3XR5kcCM",
            "_source": {
              "@timestamp": "2099-12-06T11:04:07.000Z",
              "event": {
                "category": "file",
                "id": "dGCHwoeS",
                "sequence": 2
              },
              "file": {
                "accessed": "2099-12-07T11:07:08.000Z",
                "name": "cmd.exe",
                "path": "C:\\Windows\\System32\\cmd.exe",
                "type": "file",
                "size": 16384
              },
              "process": {
                "pid": 2012,
                "name": "cmd.exe",
                "executable": "C:\\Windows\\System32\\cmd.exe"
              }
            }
          },
          {
            "_index": ".ds-my-data-stream-2099.12.07-000001",
            "_id": "OQmfCaduce8zoHT93o4H",
            "_source": {
              "@timestamp": "2099-12-07T11:07:09.000Z",
              "event": {
                "category": "process",
                "id": "aR3NWVOs",
                "sequence": 4
              },
              "process": {
                "pid": 2012,
                "name": "regsvr32.exe",
                "command_line": "regsvr32.exe  /s /u /i:https://...RegSvr32.sct scrobj.dll",
                "executable": "C:\\Windows\\System32\\regsvr32.exe"
              }
            }
          }
        ]
      }
    ]
  }
}

ES|QL

The Elasticsearch Query Language (ES|QL) provides a powerful way to filter, transform, and analyze data stored in Elasticsearch, and in the future in other runtimes.

Learn more about ES|QL




Get async ES|QL query results Generally available; Added in 8.13.0

GET /_query/async/{id}

Get the current status and available results or stored results for an ES|QL asynchronous query. If the Elasticsearch security features are enabled, only the user who first submitted the ES|QL query can retrieve the results using this API.

External documentation

Path parameters

  • id string Required

    The unique identifier of the query. A query ID is provided in the ES|QL async query API response for a query that does not complete in the designated time. A query ID is also provided when the request was submitted with the keep_on_completion parameter set to true.

Query parameters

  • drop_null_columns boolean

    Indicates whether columns that are entirely null will be removed from the columns and values portion of the results. If true, the response will include an extra section under the name all_columns which has the name of all the columns.

  • keep_alive string

    The period for which the query and its results are stored in the cluster. When this period expires, the query and its results are deleted, even if the query is still ongoing.

    Values are -1 or 0.

  • wait_for_completion_timeout string

    The period to wait for the request to finish. By default, the request waits for complete query results. If the request completes during the period specified in this parameter, complete query results are returned. Otherwise, the response returns an is_running value of true and no results.

    Values are -1 or 0.

Responses

  • 200 application/json
GET /_query/async/{id}
curl \
 --request GET 'http://api.example.com/_query/async/{id}' \
 --header "Authorization: $API_KEY"

Delete an async ES|QL query Generally available; Added in 8.13.0

DELETE /_query/async/{id}

If the query is still running, it is cancelled. Otherwise, the stored results are deleted.

If the Elasticsearch security features are enabled, only the following users can use this API to delete a query:

  • The authenticated user that submitted the original query request
  • Users with the cancel_task cluster privilege
External documentation

Path parameters

  • id string Required

    The unique identifier of the query. A query ID is provided in the ES|QL async query API response for a query that does not complete in the designated time. A query ID is also provided when the request was submitted with the keep_on_completion parameter set to true.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

DELETE /_query/async/{id}
curl \
 --request DELETE 'http://api.example.com/_query/async/{id}' \
 --header "Authorization: $API_KEY"

Stop async ES|QL query Generally available; Added in 8.18.0

POST /_query/async/{id}/stop

This API interrupts the query execution and returns the results so far. If the Elasticsearch security features are enabled, only the user who first submitted the ES|QL query can stop it.

External documentation

Path parameters

  • id string Required

    The unique identifier of the query. A query ID is provided in the ES|QL async query API response for a query that does not complete in the designated time. A query ID is also provided when the request was submitted with the keep_on_completion parameter set to true.

Query parameters

  • drop_null_columns boolean

    Indicates whether columns that are entirely null will be removed from the columns and values portion of the results. If true, the response will include an extra section under the name all_columns which has the name of all the columns.

Responses

  • 200 application/json
POST /_query/async/{id}/stop
curl \
 --request POST 'http://api.example.com/_query/async/{id}/stop' \
 --header "Authorization: $API_KEY"

Run an ES|QL query Generally available; Added in 8.11.0

POST /_query

Get search results for an ES|QL (Elasticsearch query language) query.

External documentation

Query parameters

  • format string

    A short version of the Accept header, e.g. json, yaml.

    Values are csv, json, tsv, txt, yaml, cbor, smile, or arrow.

  • delimiter string

    The character to use between values within a CSV row. Only valid for the CSV format.

  • drop_null_columns boolean

    Should columns that are entirely null be removed from the columns and values portion of the results? Defaults to false. If true then the response will include an extra section under the name all_columns which has the name of all columns.

application/json

Body Required

  • columnar boolean

    By default, ES|QL returns results as rows. For example, FROM returns each individual document as one row. For the JSON, YAML, CBOR and smile formats, ES|QL can return the results in a columnar fashion where one row represents all the values of a certain column in the results.

  • filter object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation
  • locale string
  • params array[number | string | boolean | null | object]

    To avoid any attempts of hacking or code injection, extract the values in a separate list of parameters. Use question mark placeholders (?) in the query string for each of the parameters.

    A field value.

  • profile boolean

    If provided and true the response will include an extra profile object with information on how the query was executed. This information is for human debugging and its format can change at any time but it can give some insight into the performance of each part of the query.

  • query string Required

    The ES|QL query API accepts an ES|QL query string in the query parameter, runs it, and returns the results.

  • tables object

    Tables to use with the LOOKUP operation. The top level key is the table name and the next level key is the column name.

    Hide tables attribute Show tables attribute object
  • include_ccs_metadata boolean

    When set to true and performing a cross-cluster query, the response will include an extra _clusters object with information about the clusters that participated in the search along with info such as shards count.

Responses

  • 200 application/json
POST /_query
curl \
 --request POST 'http://api.example.com/_query' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"query\": \"\"\"\n    FROM library,remote-*:library\n    | EVAL year = DATE_TRUNC(1 YEARS, release_date)\n    | STATS MAX(page_count) BY year\n    | SORT year\n    | LIMIT 5\n  \"\"\",\n  \"include_ccs_metadata\": true\n}"'
Request example
Run `POST /_query` to get results for an ES|QL query.
{
  "query": """
    FROM library,remote-*:library
    | EVAL year = DATE_TRUNC(1 YEARS, release_date)
    | STATS MAX(page_count) BY year
    | SORT year
    | LIMIT 5
  """,
  "include_ccs_metadata": true
}

Get the features Generally available; Added in 7.12.0

GET /_features

Get a list of features that can be included in snapshots using the feature_states field when creating a snapshot. You can use this API to determine which feature states to include when taking a snapshot. By default, all feature states are included in a snapshot if that snapshot includes the global state, or none if it does not.

A feature state includes one or more system indices necessary for a given feature to function. In order to ensure data integrity, all system indices that comprise a feature state are snapshotted and restored together.

The features listed by this API are a combination of built-in features and features defined by plugins. In order for a feature state to be listed in this API and recognized as a valid feature state by the create snapshot API, the plugin that defines that feature must be installed on the master node.

External documentation

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • features array[object] Required
      Hide features attributes Show features attributes object
      • name string Required
      • description string Required
GET /_features
curl \
 --request GET 'http://api.example.com/_features' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response for retrieving a list of feature states that can be included when taking a snapshot.
{
  "features": [
    {
      "name": "tasks",
      "description": "Manages task results"
    },
    {
      "name": "kibana",
      "description": "Manages Kibana configuration and reports"
    }
  ]
}




Fleet

Get global checkpoints Generally available; Added in 7.13.0

GET /{index}/_fleet/global_checkpoints

Get the current global checkpoints for an index. This API is designed for internal use by the Fleet server project.

Path parameters

  • index string Required

    A single index or index alias that resolves to a single index.

Query parameters

  • wait_for_advance boolean

    A boolean value which controls whether to wait (until the timeout) for the global checkpoints to advance past the provided checkpoints.

  • wait_for_index boolean

    A boolean value which controls whether to wait (until the timeout) for the target index to exist and all primary shards be active. Can only be true when wait_for_advance is true.

  • checkpoints array[number]

    A comma separated list of previous global checkpoints. When used in combination with wait_for_advance, the API will only return once the global checkpoints advances past the checkpoints. Providing an empty list will cause Elasticsearch to immediately return the current global checkpoints.

  • timeout string

    Period to wait for a global checkpoints to advance past checkpoints.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • global_checkpoints array[number] Required
    • timed_out boolean Required
GET /{index}/_fleet/global_checkpoints
curl \
 --request GET 'http://api.example.com/{index}/_fleet/global_checkpoints' \
 --header "Authorization: $API_KEY"

























Explore graph analytics Generally available

GET /{index}/_graph/explore

Extract and summarize information about the documents and terms in an Elasticsearch data stream or index. The easiest way to understand the behavior of this API is to use the Graph UI to explore connections. An initial request to the _explore API contains a seed query that identifies the documents of interest and specifies the fields that define the vertices and connections you want to include in the graph. Subsequent requests enable you to spider out from one more vertices of interest. You can exclude vertices that have already been returned.

External documentation

Path parameters

  • index string | array[string] Required

    Name of the index.

Query parameters

  • routing string

    Custom value used to route operations to a specific shard.

  • timeout string

    Specifies the period of time to wait for a response from each shard. If no response is received before the timeout expires, the request fails and returns an error. Defaults to no timeout.

    Values are -1 or 0.

application/json

Body

  • connections object
    Hide connections attributes Show connections attributes object
    • connections object
    • query object Required

      An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

      External documentation
    • vertices array[object] Required

      Contains the fields you are interested in.

      Hide vertices attributes Show vertices attributes object
      • exclude array[string]

        Prevents the specified terms from being included in the results.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • include array[object]

        Identifies the terms of interest that form the starting points from which you want to spider out.

        Hide include attributes Show include attributes object
        • boost number Required
        • term string Required
      • min_doc_count number

        Specifies how many documents must contain a pair of terms before it is considered to be a useful connection. This setting acts as a certainty threshold.

      • shard_min_doc_count number

        Controls how many documents on a particular shard have to contain a pair of terms before the connection is returned for global consideration.

      • size number

        Specifies the maximum number of vertex terms returned for each field.

  • controls object
    Hide controls attributes Show controls attributes object
    • sample_diversity object
      Hide sample_diversity attributes Show sample_diversity attributes object
      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • max_docs_per_value number Required
    • sample_size number

      Each hop considers a sample of the best-matching documents on each shard. Using samples improves the speed of execution and keeps exploration focused on meaningfully-connected terms. Very small values (less than 50) might not provide sufficient weight-of-evidence to identify significant connections between terms. Very large sample sizes can dilute the quality of the results and increase execution times.

    • timeout string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • use_significance boolean Required

      Filters associated terms so only those that are significantly associated with your query are included.

  • query object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation
  • vertices array[object]

    Specifies one or more fields that contain the terms you want to include in the graph as vertices.

    Hide vertices attributes Show vertices attributes object
    • exclude array[string]

      Prevents the specified terms from being included in the results.

    • field string Required

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • include array[object]

      Identifies the terms of interest that form the starting points from which you want to spider out.

      Hide include attributes Show include attributes object
      • boost number Required
      • term string Required
    • min_doc_count number

      Specifies how many documents must contain a pair of terms before it is considered to be a useful connection. This setting acts as a certainty threshold.

    • shard_min_doc_count number

      Controls how many documents on a particular shard have to contain a pair of terms before the connection is returned for global consideration.

    • size number

      Specifies the maximum number of vertex terms returned for each field.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • connections array[object] Required
      Hide connections attributes Show connections attributes object
      • doc_count number Required
      • source number Required
      • target number Required
      • weight number Required
    • failures array[object] Required
      Hide failures attributes Show failures attributes object
      • index string
      • node string
      • reason object Required

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide reason attributes Show reason attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • shard number Required
      • status string
    • timed_out boolean Required
    • took number Required
    • vertices array[object] Required
      Hide vertices attributes Show vertices attributes object
      • depth number Required
      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • term string Required
      • weight number Required
GET /{index}/_graph/explore
curl \
 --request GET 'http://api.example.com/{index}/_graph/explore' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"query\": {\n    \"match\": {\n      \"query.raw\": \"midi\"\n    }\n  },\n  \"vertices\": [\n    {\n      \"field\": \"product\"\n    }\n  ],\n  \"connections\": {\n    \"vertices\": [\n      {\n        \"field\": \"query.raw\"\n      }\n    ]\n  }\n}"'
Request example
Run `POST clicklogs/_graph/explore` for a basic exploration An initial graph explore query typically begins with a query to identify strongly related terms. Seed the exploration with a query. This example is searching `clicklogs` for people who searched for the term `midi`.Identify the vertices to include in the graph. This example is looking for product codes that are significantly associated with searches for `midi`. Find the connections. This example is looking for other search terms that led people to click on the products that are associated with searches for `midi`.
{
  "query": {
    "match": {
      "query.raw": "midi"
    }
  },
  "vertices": [
    {
      "field": "product"
    }
  ],
  "connections": {
    "vertices": [
      {
        "field": "query.raw"
      }
    ]
  }
}

Explore graph analytics Generally available

POST /{index}/_graph/explore

Extract and summarize information about the documents and terms in an Elasticsearch data stream or index. The easiest way to understand the behavior of this API is to use the Graph UI to explore connections. An initial request to the _explore API contains a seed query that identifies the documents of interest and specifies the fields that define the vertices and connections you want to include in the graph. Subsequent requests enable you to spider out from one more vertices of interest. You can exclude vertices that have already been returned.

External documentation

Path parameters

  • index string | array[string] Required

    Name of the index.

Query parameters

  • routing string

    Custom value used to route operations to a specific shard.

  • timeout string

    Specifies the period of time to wait for a response from each shard. If no response is received before the timeout expires, the request fails and returns an error. Defaults to no timeout.

    Values are -1 or 0.

application/json

Body

  • connections object
    Hide connections attributes Show connections attributes object
    • connections object
    • query object Required

      An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

      External documentation
    • vertices array[object] Required

      Contains the fields you are interested in.

      Hide vertices attributes Show vertices attributes object
      • exclude array[string]

        Prevents the specified terms from being included in the results.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • include array[object]

        Identifies the terms of interest that form the starting points from which you want to spider out.

        Hide include attributes Show include attributes object
        • boost number Required
        • term string Required
      • min_doc_count number

        Specifies how many documents must contain a pair of terms before it is considered to be a useful connection. This setting acts as a certainty threshold.

      • shard_min_doc_count number

        Controls how many documents on a particular shard have to contain a pair of terms before the connection is returned for global consideration.

      • size number

        Specifies the maximum number of vertex terms returned for each field.

  • controls object
    Hide controls attributes Show controls attributes object
    • sample_diversity object
      Hide sample_diversity attributes Show sample_diversity attributes object
      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • max_docs_per_value number Required
    • sample_size number

      Each hop considers a sample of the best-matching documents on each shard. Using samples improves the speed of execution and keeps exploration focused on meaningfully-connected terms. Very small values (less than 50) might not provide sufficient weight-of-evidence to identify significant connections between terms. Very large sample sizes can dilute the quality of the results and increase execution times.

    • timeout string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • use_significance boolean Required

      Filters associated terms so only those that are significantly associated with your query are included.

  • query object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation
  • vertices array[object]

    Specifies one or more fields that contain the terms you want to include in the graph as vertices.

    Hide vertices attributes Show vertices attributes object
    • exclude array[string]

      Prevents the specified terms from being included in the results.

    • field string Required

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • include array[object]

      Identifies the terms of interest that form the starting points from which you want to spider out.

      Hide include attributes Show include attributes object
      • boost number Required
      • term string Required
    • min_doc_count number

      Specifies how many documents must contain a pair of terms before it is considered to be a useful connection. This setting acts as a certainty threshold.

    • shard_min_doc_count number

      Controls how many documents on a particular shard have to contain a pair of terms before the connection is returned for global consideration.

    • size number

      Specifies the maximum number of vertex terms returned for each field.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • connections array[object] Required
      Hide connections attributes Show connections attributes object
      • doc_count number Required
      • source number Required
      • target number Required
      • weight number Required
    • failures array[object] Required
      Hide failures attributes Show failures attributes object
      • index string
      • node string
      • reason object Required

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide reason attributes Show reason attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • shard number Required
      • status string
    • timed_out boolean Required
    • took number Required
    • vertices array[object] Required
      Hide vertices attributes Show vertices attributes object
      • depth number Required
      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • term string Required
      • weight number Required
POST /{index}/_graph/explore
curl \
 --request POST 'http://api.example.com/{index}/_graph/explore' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"query\": {\n    \"match\": {\n      \"query.raw\": \"midi\"\n    }\n  },\n  \"vertices\": [\n    {\n      \"field\": \"product\"\n    }\n  ],\n  \"connections\": {\n    \"vertices\": [\n      {\n        \"field\": \"query.raw\"\n      }\n    ]\n  }\n}"'
Request example
Run `POST clicklogs/_graph/explore` for a basic exploration An initial graph explore query typically begins with a query to identify strongly related terms. Seed the exploration with a query. This example is searching `clicklogs` for people who searched for the term `midi`.Identify the vertices to include in the graph. This example is looking for product codes that are significantly associated with searches for `midi`. Find the connections. This example is looking for other search terms that led people to click on the products that are associated with searches for `midi`.
{
  "query": {
    "match": {
      "query.raw": "midi"
    }
  },
  "vertices": [
    {
      "field": "product"
    }
  ],
  "connections": {
    "vertices": [
      {
        "field": "query.raw"
      }
    ]
  }
}

Index

Index APIs enable you to manage individual indices, index settings, aliases, mappings, and index templates.





Create or update a component template Generally available; Added in 7.8.0

PUT /_component_template/{name}

Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.

An index template can be composed of multiple component templates. To use a component template, specify it in an index template’s composed_of list. Component templates are only applied to new data streams and indices as part of a matching index template.

Settings and mappings specified directly in the index template or the create index request override any settings or mappings specified in a component template.

Component templates are only used during index creation. For data streams, this includes data stream creation and the creation of a stream’s backing indices. Changes to component templates do not affect existing indices, including a stream’s backing indices.

You can use C-style /* *\/ block comments in component templates. You can include comments anywhere in the request body except before the opening curly bracket.

Applying component templates

You cannot directly apply a component template to a data stream or index. To be applied, a component template must be included in an index template's composed_of list.

Required authorization

  • Cluster privileges: manage_index_templates

Path parameters

  • name string Required

    Name of the component template to create. Elasticsearch includes the following built-in component templates: logs-mappings; logs-settings; metrics-mappings; metrics-settings;synthetics-mapping; synthetics-settings. Elastic Agent uses these templates to configure backing indices for its data streams. If you use Elastic Agent and want to overwrite one of these templates, set the version for your replacement template higher than the current version. If you don’t use Elastic Agent and want to disable all built-in component and index templates, set stack.templates.enabled to false using the cluster update settings API.

Query parameters

  • create boolean

    If true, this request cannot replace or update existing component templates.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

application/json

Body Required

  • template object Required
    Hide template attributes Show template attributes object
    • aliases object
      Hide aliases attribute Show aliases attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • filter object

          An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

          External documentation
        • index_routing string
        • is_hidden boolean

          If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

        • is_write_index boolean

          If true, the index is the write index for the alias.

        • routing string
        • search_routing string
    • mappings object
      Hide mappings attributes Show mappings attributes object
      • all_field object
        Hide all_field attributes Show all_field attributes object
        • analyzer string Required
        • enabled boolean Required
        • omit_norms boolean Required
        • search_analyzer string Required
        • similarity string Required
        • store boolean Required
        • store_term_vector_offsets boolean Required
        • store_term_vector_payloads boolean Required
        • store_term_vector_positions boolean Required
        • store_term_vectors boolean Required
      • date_detection boolean
      • dynamic string

        Values are strict, runtime, true, or false.

      • dynamic_date_formats array[string]
      • dynamic_templates array[object]
      • _field_names object
        Hide _field_names attribute Show _field_names attribute object
        • enabled boolean Required
      • index_field object
        Hide index_field attribute Show index_field attribute object
        • enabled boolean Required
      • _meta object
        Hide _meta attribute Show _meta attribute object
        • * object Additional properties
      • numeric_detection boolean
      • properties object
      • _routing object
        Hide _routing attribute Show _routing attribute object
        • required boolean Required
      • _size object
        Hide _size attribute Show _size attribute object
        • enabled boolean Required
      • _source object
        Hide _source attributes Show _source attributes object
        • compress boolean
        • compress_threshold string
        • enabled boolean
        • excludes array[string]
        • includes array[string]
        • mode string

          Values are disabled, stored, or synthetic.

      • runtime object
        Hide runtime attribute Show runtime attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • fields object

            For type composite

            Hide fields attribute Show fields attribute object
            • * object Additional properties
              Hide * attribute Show * attribute object
              • type string Required

                Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

          • fetch_fields array[object]

            For type lookup

            Hide fetch_fields attributes Show fetch_fields attributes object
            • field string Required

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • format string
          • format string

            A custom format for date type runtime fields.

          • input_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_index string
          • script object
            Hide script attributes Show script attributes object
            • source string

              The script source.

            • id string
            • params object

              Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

              Hide params attribute Show params attribute object
              • * object Additional properties
            • lang string

              Any of:

              Values are painless, expression, mustache, or java.

            • options object
              Hide options attribute Show options attribute object
              • * string Additional properties
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • enabled boolean
      • subobjects string

        Values are true or false.

      • _data_stream_timestamp object
        Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
        • enabled boolean Required
    • settings object
      Index settings
    • defaults object
      Index settings
    • data_stream string
    • lifecycle object

      Data stream lifecycle denotes that a data stream is managed by the data stream lifecycle and contains the configuration.

      Hide lifecycle attributes Show lifecycle attributes object
      • data_retention string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • downsampling object
        Hide downsampling attribute Show downsampling attribute object
        • rounds array[object] Required

          The list of downsampling rounds to execute as part of this downsampling configuration

          Hide rounds attributes Show rounds attributes object
          • after string Required

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • config object Required
            Hide config attribute Show config attribute object
            • fixed_interval string Required

              A date histogram interval. Similar to Duration with additional units: w (week), M (month), q (quarter) and y (year)

      • enabled boolean

        If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

  • version number
  • _meta object
    Hide _meta attribute Show _meta attribute object
    • * object Additional properties
  • deprecated boolean

    Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_component_template/{name}
curl \
 --request PUT 'http://api.example.com/_component_template/{name}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"template\": null,\n  \"settings\": {\n    \"number_of_shards\": 1\n  },\n  \"mappings\": {\n    \"_source\": {\n      \"enabled\": false\n    },\n    \"properties\": {\n      \"host_name\": {\n        \"type\": \"keyword\"\n      },\n      \"created_at\": {\n        \"type\": \"date\",\n        \"format\": \"EEE MMM dd HH:mm:ss Z yyyy\"\n      }\n    }\n  }\n}"'
Request examples
{
  "template": null,
  "settings": {
    "number_of_shards": 1
  },
  "mappings": {
    "_source": {
      "enabled": false
    },
    "properties": {
      "host_name": {
        "type": "keyword"
      },
      "created_at": {
        "type": "date",
        "format": "EEE MMM dd HH:mm:ss Z yyyy"
      }
    }
  }
}
You can include index aliases in a component template. During index creation, the `{index}` placeholder in the alias name will be replaced with the actual index name that the template gets applied to.
{
  "template": null,
  "settings": {
    "number_of_shards": 1
  },
  "aliases": {
    "alias1": {},
    "alias2": {
      "filter": {
        "term": {
          "user.id": "kimchy"
        }
      },
      "routing": "shard-1"
    },
    "{index}-alias": {}
  }
}




Delete component templates Generally available; Added in 7.8.0

DELETE /_component_template/{name}

Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.

Required authorization

  • Cluster privileges: manage_index_templates

Path parameters

  • name string | array[string] Required

    Comma-separated list or wildcard expression of component template names used to limit the request.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

DELETE /_component_template/{name}
curl \
 --request DELETE 'http://api.example.com/_component_template/{name}' \
 --header "Authorization: $API_KEY"

Check component templates Generally available; Added in 7.8.0

HEAD /_component_template/{name}

Returns information about whether a particular component template exists.

Path parameters

  • name string | array[string] Required

    Comma-separated list of component template names used to limit the request. Wildcard (*) expressions are supported.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • local boolean

    If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.

Responses

  • 200 application/json
HEAD /_component_template/{name}
curl \
 --request HEAD 'http://api.example.com/_component_template/{name}' \
 --header "Authorization: $API_KEY"












Get the dangling indices Generally available; Added in 7.9.0

GET /_dangling

If Elasticsearch encounters index data that is absent from the current cluster state, those indices are considered to be dangling. For example, this can happen if you delete more than cluster.indices.tombstones.size indices while an Elasticsearch node is offline.

Use this API to list dangling indices, which you can then import or delete.

Required authorization

  • Cluster privileges: manage

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • dangling_indices array[object] Required
      Hide dangling_indices attributes Show dangling_indices attributes object
      • index_name string Required
      • index_uuid string Required
      • creation_date_millis number

        Time unit for milliseconds

      • node_ids string | array[string] Required

GET /_dangling
curl \
 --request GET 'http://api.example.com/_dangling' \
 --header "Authorization: $API_KEY"
Response examples (200)
{
  "dangling_indices": [
   {
    "index_name": "my-index-000001",
    "index_uuid": "zmM4e0JtBkeUjiHD-MihPQ",
    "creation_date_millis": 1589414451372,
    "node_ids": [
      "pL47UN3dAb2d5RCWP6lQ3e"
    ]
   }
  ]
}

Add an index block Generally available; Added in 7.9.0

PUT /{index}/_block/{block}

Add an index block to an index. Index blocks limit the operations allowed on an index by blocking specific operation types.

Path parameters

  • index string Required

    A comma-separated list or wildcard expression of index names used to limit the request. By default, you must explicitly name the indices you are adding blocks to. To allow the adding of blocks to indices with _all, *, or other wildcard expressions, change the action.destructive_requires_name setting to false. You can update this setting in the elasticsearch.yml file or by using the cluster update settings API.

  • block string Required

    The block type to add to the index.

    Values are metadata, read, read_only, or write.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    The type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. It supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • master_timeout string

    The period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

    Values are -1 or 0.

  • timeout string

    The period to wait for a response from all relevant nodes in the cluster after updating the cluster metadata. If no response is received before the timeout expires, the cluster metadata update still applies but the response will indicate that it was not completely acknowledged. It can also be set to -1 to indicate that the request should never timeout.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required
    • shards_acknowledged boolean Required
    • indices array[object] Required
      Hide indices attributes Show indices attributes object
      • name string Required
      • blocked boolean Required
PUT /{index}/_block/{block}
curl \
 --request PUT 'http://api.example.com/{index}/_block/{block}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `PUT /my-index-000001/_block/write`, which adds an index block to an index.'
{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "indices" : [ {
    "name" : "my-index-000001",
    "blocked" : true
  } ]
}








Get tokens from text analysis Generally available

GET /{index}/_analyze

The analyze API performs analysis on a text string and returns the resulting tokens.

Generating excessive amount of tokens may cause a node to run out of memory. The index.analyze.max_token_count setting enables you to limit the number of tokens that can be produced. If more than this limit of tokens gets generated, an error occurs. The _analyze endpoint without a specified index will always use 10000 as its limit.

Required authorization

  • Index privileges: index
External documentation

Path parameters

  • index string Required

    Index used to derive the analyzer. If specified, the analyzer or field parameter overrides this value. If no index is specified or the index does not have a default analyzer, the analyze API uses the standard analyzer.

Query parameters

  • index string

    Index used to derive the analyzer. If specified, the analyzer or field parameter overrides this value. If no index is specified or the index does not have a default analyzer, the analyze API uses the standard analyzer.

application/json

Body

  • analyzer string

    The name of the analyzer that should be applied to the provided text. This could be a built-in analyzer, or an analyzer that’s been configured in the index.

  • attributes array[string]

    Array of token attributes used to filter the output of the explain parameter.

  • char_filter array

    Array of character filters used to preprocess characters before the tokenizer.

    External documentation
  • explain boolean

    If true, the response includes token attributes and additional details.

  • field string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • filter array

    Array of token filters used to apply after the tokenizer.

    External documentation
  • normalizer string

    Normalizer to use to convert text into a single token.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • detail object
      Hide detail attributes Show detail attributes object
      • analyzer object
        Hide analyzer attributes Show analyzer attributes object
        • name string Required
        • tokens array[object] Required
          Hide tokens attributes Show tokens attributes object
          • bytes string Required
          • end_offset number Required
          • keyword boolean
          • position number Required
          • positionLength number Required
          • start_offset number Required
          • termFrequency number Required
          • token string Required
          • type string Required
      • charfilters array[object]
        Hide charfilters attributes Show charfilters attributes object
        • filtered_text array[string] Required
        • name string Required
      • custom_analyzer boolean Required
      • tokenfilters array[object]
        Hide tokenfilters attributes Show tokenfilters attributes object
        • name string Required
        • tokens array[object] Required
          Hide tokens attributes Show tokens attributes object
          • bytes string Required
          • end_offset number Required
          • keyword boolean
          • position number Required
          • positionLength number Required
          • start_offset number Required
          • termFrequency number Required
          • token string Required
          • type string Required
      • tokenizer object
        Hide tokenizer attributes Show tokenizer attributes object
        • name string Required
        • tokens array[object] Required
          Hide tokens attributes Show tokens attributes object
          • bytes string Required
          • end_offset number Required
          • keyword boolean
          • position number Required
          • positionLength number Required
          • start_offset number Required
          • termFrequency number Required
          • token string Required
          • type string Required
    • tokens array[object]
      Hide tokens attributes Show tokens attributes object
      • end_offset number Required
      • position number Required
      • positionLength number
      • start_offset number Required
      • token string Required
      • type string Required
GET /{index}/_analyze
curl \
 --request GET 'http://api.example.com/{index}/_analyze' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"analyzer\": \"standard\",\n  \"text\": \"this is a test\"\n}"'
You can apply any of the built-in analyzers to the text string without specifying an index.
{
  "analyzer": "standard",
  "text": "this is a test"
}
If the text parameter is provided as array of strings, it is analyzed as a multi-value field.
{
  "analyzer": "standard",
  "text": [
    "this is a test",
    "the second text"
  ]
}
You can test a custom transient analyzer built from tokenizers, token filters, and char filters. Token filters use the filter parameter.
{
  "tokenizer": "keyword",
  "filter": [
    "lowercase"
  ],
  "char_filter": [
    "html_strip"
  ],
  "text": "this is a <b>test</b>"
}
Custom tokenizers, token filters, and character filters can be specified in the request body.
{
  "tokenizer": "whitespace",
  "filter": [
    "lowercase",
    {
      "type": "stop",
      "stopwords": [
        "a",
        "is",
        "this"
      ]
    }
  ],
  "text": "this is a test"
}
Run `GET /analyze_sample/_analyze` to run an analysis on the text using the default index analyzer associated with the `analyze_sample` index. Alternatively, the analyzer can be derived based on a field mapping.
{
  "field": "obj1.field1",
  "text": "this is a test"
}
Run `GET /analyze_sample/_analyze` and supply a normalizer for a keyword field if there is a normalizer associated with the specified index.
{
  "normalizer": "my_normalizer",
  "text": "BaR"
}
If you want to get more advanced details, set `explain` to `true`. It will output all token attributes for each token. You can filter token attributes you want to output by setting the `attributes` option. NOTE: The format of the additional detail information is labelled as experimental in Lucene and it may change in the future.
{
  "tokenizer": "standard",
  "filter": [
    "snowball"
  ],
  "text": "detailed output",
  "explain": true,
  "attributes": [
    "keyword"
  ]
}
Response examples (200)
A successful response for an analysis with `explain` set to `true`.
{
  "detail": {
    "custom_analyzer": true,
    "charfilters": [],
    "tokenizer": {
      "name": "standard",
      "tokens": [
        {
          "token": "detailed",
          "start_offset": 0,
          "end_offset": 8,
          "type": "<ALPHANUM>",
          "position": 0
        },
        {
          "token": "output",
          "start_offset": 9,
          "end_offset": 15,
          "type": "<ALPHANUM>",
          "position": 1
        }
      ]
    },
    "tokenfilters": [
      {
        "name": "snowball",
        "tokens": [
          {
            "token": "detail",
            "start_offset": 0,
            "end_offset": 8,
            "type": "<ALPHANUM>",
            "position": 0,
            "keyword": false
          },
          {
            "token": "output",
            "start_offset": 9,
            "end_offset": 15,
            "type": "<ALPHANUM>",
            "position": 1,
            "keyword": false
          }
        ]
      }
    ]
  }
}




Clear the cache Generally available

POST /_cache/clear

Clear the cache of one or more indices. For data streams, the API clears the caches of the stream's backing indices.

By default, the clear cache API clears all caches. To clear only specific caches, use the fielddata, query, or request parameters. To clear the cache only of specific fields, use the fields parameter.

Required authorization

  • Index privileges: manage

Query parameters

  • index string | array[string]

    Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • fielddata boolean

    If true, clears the fields cache. Use the fields parameter to clear the cache of specific fields only.

  • fields string | array[string]

    Comma-separated list of field names used to limit the fielddata parameter.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • query boolean

    If true, clears the query cache.

  • request boolean

    If true, clears the request cache.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • _shards object
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number Required
        • status string
      • skipped number
POST /_cache/clear
curl \
 --request POST 'http://api.example.com/_cache/clear' \
 --header "Authorization: $API_KEY"








Clone an index Generally available; Added in 7.4.0

POST /{index}/_clone/{target}

Clone an existing index into a new index. Each original primary shard is cloned into a new primary shard in the new index.

IMPORTANT: Elasticsearch does not apply index templates to the resulting index. The API also does not copy index metadata from the original index. Index metadata includes aliases, index lifecycle management phase definitions, and cross-cluster replication (CCR) follower information. For example, if you clone a CCR follower index, the resulting clone will not be a follower index.

The clone API copies most index settings from the source index to the resulting index, with the exception of index.number_of_replicas and index.auto_expand_replicas. To set the number of replicas in the resulting index, configure these settings in the clone request.

Cloning works as follows:

  • First, it creates a new target index with the same definition as the source index.
  • Then it hard-links segments from the source index into the target index. If the file system does not support hard-linking, all segments are copied into the new index, which is a much more time consuming process.
  • Finally, it recovers the target index as though it were a closed index which had just been re-opened.

IMPORTANT: Indices can only be cloned if they meet the following requirements:

  • The index must be marked as read-only and have a cluster health status of green.
  • The target index must not exist.
  • The source index must have the same number of primary shards as the target index.
  • The node handling the clone process must have sufficient free disk space to accommodate a second copy of the existing index.

The current write index on a data stream cannot be cloned. In order to clone the current write index, the data stream must first be rolled over so that a new write index is created and then the previous write index can be cloned.

NOTE: Mappings cannot be specified in the _clone request. The mappings of the source index will be used for the target index.

Monitor the cloning process

The cloning process can be monitored with the cat recovery API or the cluster health API can be used to wait until all primary shards have been allocated by setting the wait_for_status parameter to yellow.

The _clone API returns as soon as the target index has been added to the cluster state, before any shards have been allocated. At this point, all shards are in the state unassigned. If, for any reason, the target index can't be allocated, its primary shard will remain unassigned until it can be allocated on that node.

Once the primary shard is allocated, it moves to state initializing, and the clone process begins. When the clone operation completes, the shard will become active. At that point, Elasticsearch will try to allocate any replicas and may decide to relocate the primary shard to another node.

Wait for active shards

Because the clone operation creates a new index to clone the shards to, the wait for active shards setting on index creation applies to the clone index action as well.

Required authorization

  • Index privileges: manage

Path parameters

  • index string Required

    Name of the source index to clone.

  • target string Required

    Name of the target index to create.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).

    Values are all or index-setting.

application/json

Body

  • aliases object

    Aliases for the resulting index.

    Hide aliases attribute Show aliases attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • filter object

        An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

        External documentation
      • index_routing string
      • is_hidden boolean

        If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

      • is_write_index boolean

        If true, the index is the write index for the alias.

      • routing string
      • search_routing string
  • settings object

    Configuration options for the target index.

    Hide settings attribute Show settings attribute object
    • * object Additional properties

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required
    • index string Required
    • shards_acknowledged boolean Required
POST /{index}/_clone/{target}
curl \
 --request POST 'http://api.example.com/{index}/_clone/{target}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"settings\": {\n    \"index.number_of_shards\": 5\n  },\n  \"aliases\": {\n    \"my_search_indices\": {}\n  }\n}"'
Request example
Clone `my_source_index` into a new index called `my_target_index` with `POST /my_source_index/_clone/my_target_index`. The API accepts `settings` and `aliases` parameters for the target index.
{
  "settings": {
    "index.number_of_shards": 5
  },
  "aliases": {
    "my_search_indices": {}
  }
}








Create an index Generally available

PUT /{index}

You can use the create index API to add a new index to an Elasticsearch cluster. When creating an index, you can specify the following:

  • Settings for the index.
  • Mappings for fields in the index.
  • Index aliases

Wait for active shards

By default, index creation will only return a response to the client when the primary copies of each shard have been started, or the request times out. The index creation response will indicate what happened. For example, acknowledged indicates whether the index was successfully created in the cluster, while shards_acknowledged indicates whether the requisite number of shard copies were started for each shard in the index before timing out. Note that it is still possible for either acknowledged or shards_acknowledged to be false, but for the index creation to be successful. These values simply indicate whether the operation completed before the timeout. If acknowledged is false, the request timed out before the cluster state was updated with the newly created index, but it probably will be created sometime soon. If shards_acknowledged is false, then the request timed out before the requisite number of shards were started (by default just the primaries), even if the cluster state was successfully updated to reflect the newly created index (that is to say, acknowledged is true).

You can change the default of only waiting for the primary shards to start through the index setting index.write.wait_for_active_shards. Note that changing this setting will also affect the wait_for_active_shards value on all subsequent write operations.

Required authorization

  • Index privileges: create_index,manage

Path parameters

  • index string Required

    Name of the index you wish to create.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).

    Values are all or index-setting.

application/json

Body

  • aliases object

    Aliases for the index.

    Hide aliases attribute Show aliases attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • filter object

        An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

        External documentation
      • index_routing string
      • is_hidden boolean

        If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

      • is_write_index boolean

        If true, the index is the write index for the alias.

      • routing string
      • search_routing string
  • mappings object
    Hide mappings attributes Show mappings attributes object
    • all_field object
      Hide all_field attributes Show all_field attributes object
      • analyzer string Required
      • enabled boolean Required
      • omit_norms boolean Required
      • search_analyzer string Required
      • similarity string Required
      • store boolean Required
      • store_term_vector_offsets boolean Required
      • store_term_vector_payloads boolean Required
      • store_term_vector_positions boolean Required
      • store_term_vectors boolean Required
    • date_detection boolean
    • dynamic string

      Values are strict, runtime, true, or false.

    • dynamic_date_formats array[string]
    • dynamic_templates array[object]
    • _field_names object
      Hide _field_names attribute Show _field_names attribute object
      • enabled boolean Required
    • index_field object
      Hide index_field attribute Show index_field attribute object
      • enabled boolean Required
    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties
    • numeric_detection boolean
    • properties object
    • _routing object
      Hide _routing attribute Show _routing attribute object
      • required boolean Required
    • _size object
      Hide _size attribute Show _size attribute object
      • enabled boolean Required
    • _source object
      Hide _source attributes Show _source attributes object
      • compress boolean
      • compress_threshold string
      • enabled boolean
      • excludes array[string]
      • includes array[string]
      • mode string

        Values are disabled, stored, or synthetic.

    • runtime object
      Hide runtime attribute Show runtime attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • format string
        • format string

          A custom format for date type runtime fields.

        • input_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_index string
        • script object
          Hide script attributes Show script attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties
          • lang string

            Any of:

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • type string Required

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • enabled boolean
    • subobjects string

      Values are true or false.

    • _data_stream_timestamp object
      Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
      • enabled boolean Required
  • settings object
    Index settings

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • index string Required
    • shards_acknowledged boolean Required
    • acknowledged boolean Required
PUT /{index}
curl \
 --request PUT 'http://api.example.com/{index}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"settings\": {\n    \"number_of_shards\": 3,\n    \"number_of_replicas\": 2\n  }\n}"'
This request specifies the `number_of_shards` and `number_of_replicas`.
{
  "settings": {
    "number_of_shards": 3,
    "number_of_replicas": 2
  }
}
You can provide mapping definitions in the create index API requests.
{
  "settings": {
    "number_of_shards": 1
  },
  "mappings": {
    "properties": {
      "field1": { "type": "text" }
    }
  }
}
You can provide mapping definitions in the create index API requests. Index alias names also support date math.
{
  "aliases": {
    "alias_1": {},
    "alias_2": {
      "filter": {
        "term": {
          "user.id": "kimchy"
        }
      },
      "routing": "shard-1"
    }
  }
}












Create or update an alias Generally available

PUT /{index}/_alias/{name}

Adds a data stream or index to an alias.

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams or indices to add. Supports wildcards (*). Wildcard patterns that match both data streams and indices return an error.

  • name string Required

    Alias to update. If the alias doesn’t exist, the request creates it. Index alias names support date math.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

application/json

Body

  • filter object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation
  • index_routing string
  • is_write_index boolean

    If true, sets the write index or data stream for the alias. If an alias points to multiple indices or data streams and is_write_index isn’t set, the alias rejects write requests. If an index alias points to one index and is_write_index isn’t set, the index automatically acts as the write index. Data stream aliases don’t automatically set a write data stream, even if the alias points to one data stream.

  • routing string
  • search_routing string

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /{index}/_alias/{name}
curl \
 --request PUT 'http://api.example.com/{index}/_alias/{name}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"actions\": [\n    {\n      \"add\": {\n        \"index\": \"my-data-stream\",\n        \"alias\": \"my-alias\"\n      }\n    }\n  ]\n}"'
Request example
{
  "actions": [
    {
      "add": {
        "index": "my-data-stream",
        "alias": "my-alias"
      }
    }
  ]
}








Check aliases Generally available

HEAD /{index}/_alias/{name}

Check if one or more data stream or index aliases exist.

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams or indices used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

  • name string | array[string] Required

    Comma-separated list of aliases to check. Supports wildcards (*).

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If false, requests that include a missing data stream or index in the target indices or data streams return an error.

  • local boolean Deprecated

    If true, the request retrieves information from the local node only.

Responses

  • 200 application/json
HEAD /{index}/_alias/{name}
curl \
 --request HEAD 'http://api.example.com/{index}/_alias/{name}' \
 --header "Authorization: $API_KEY"

Create or update an alias Generally available

PUT /{index}/_aliases/{name}

Adds a data stream or index to an alias.

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams or indices to add. Supports wildcards (*). Wildcard patterns that match both data streams and indices return an error.

  • name string Required

    Alias to update. If the alias doesn’t exist, the request creates it. Index alias names support date math.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

application/json

Body

  • filter object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation
  • index_routing string
  • is_write_index boolean

    If true, sets the write index or data stream for the alias. If an alias points to multiple indices or data streams and is_write_index isn’t set, the alias rejects write requests. If an index alias points to one index and is_write_index isn’t set, the index automatically acts as the write index. Data stream aliases don’t automatically set a write data stream, even if the alias points to one data stream.

  • routing string
  • search_routing string

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /{index}/_aliases/{name}
curl \
 --request PUT 'http://api.example.com/{index}/_aliases/{name}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"actions\": [\n    {\n      \"add\": {\n        \"index\": \"my-data-stream\",\n        \"alias\": \"my-alias\"\n      }\n    }\n  ]\n}"'
Request example
{
  "actions": [
    {
      "add": {
        "index": "my-data-stream",
        "alias": "my-alias"
      }
    }
  ]
}

Create or update an alias Generally available

POST /{index}/_aliases/{name}

Adds a data stream or index to an alias.

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams or indices to add. Supports wildcards (*). Wildcard patterns that match both data streams and indices return an error.

  • name string Required

    Alias to update. If the alias doesn’t exist, the request creates it. Index alias names support date math.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

application/json

Body

  • filter object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation
  • index_routing string
  • is_write_index boolean

    If true, sets the write index or data stream for the alias. If an alias points to multiple indices or data streams and is_write_index isn’t set, the alias rejects write requests. If an index alias points to one index and is_write_index isn’t set, the index automatically acts as the write index. Data stream aliases don’t automatically set a write data stream, even if the alias points to one data stream.

  • routing string
  • search_routing string

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /{index}/_aliases/{name}
curl \
 --request POST 'http://api.example.com/{index}/_aliases/{name}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"actions\": [\n    {\n      \"add\": {\n        \"index\": \"my-data-stream\",\n        \"alias\": \"my-alias\"\n      }\n    }\n  ]\n}"'
Request example
{
  "actions": [
    {
      "add": {
        "index": "my-data-stream",
        "alias": "my-alias"
      }
    }
  ]
}








Get index templates Generally available; Added in 7.9.0

GET /_index_template/{name}

Get information about one or more index templates.

Required authorization

  • Cluster privileges: manage_index_templates

Path parameters

  • name string Required

    Comma-separated list of index template names used to limit the request. Wildcard (*) expressions are supported.

Query parameters

  • local boolean

    If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.

  • flat_settings boolean

    If true, returns settings in flat format.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • include_defaults boolean

    If true, returns all relevant default configurations for the index template.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • index_templates array[object] Required
      Hide index_templates attributes Show index_templates attributes object
      • name string Required
      • index_template object Required
        Hide index_template attributes Show index_template attributes object
        • index_patterns string | array[string] Required
        • composed_of array[string] Required

          An ordered list of component template names. Component templates are merged in the order specified, meaning that the last component template specified has the highest precedence.

        • template object
          Hide template attributes Show template attributes object
          • aliases object

            Aliases to add. If the index template includes a data_stream object, these are data stream aliases. Otherwise, these are index aliases. Data stream aliases ignore the index_routing, routing, and search_routing options.

            Hide aliases attribute Show aliases attribute object
            • * object Additional properties
          • mappings object
            Hide mappings attributes Show mappings attributes object
            • all_field object
            • date_detection boolean
            • dynamic string

              Values are strict, runtime, true, or false.

            • dynamic_date_formats array[string]
            • dynamic_templates array[object]
            • _field_names object
            • index_field object
            • _meta object
            • numeric_detection boolean
            • properties object
            • _routing object
            • _size object
            • _source object
            • runtime object
            • enabled boolean
            • subobjects string

              Values are true or false.

            • _data_stream_timestamp object
          • settings object
            Index settings
          • lifecycle object
        • version number
        • priority number

          Priority to determine index template precedence when a new data stream or index is created. The index template with the highest priority is chosen. If no priority is specified the template is treated as though it is of priority 0 (lowest priority). This number is not automatically generated by Elasticsearch.

        • _meta object
          Hide _meta attribute Show _meta attribute object
          • * object Additional properties
        • allow_auto_create boolean
        • data_stream object
          Hide data_stream attributes Show data_stream attributes object
          • hidden boolean

            If true, the data stream is hidden.

          • allow_custom_routing boolean

            If true, the data stream supports custom routing.

        • deprecated boolean Generally available; Added in 8.12.0

          Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.

        • ignore_missing_component_templates string | array[string]
GET /_index_template/{name}
curl \
 --request GET 'http://api.example.com/_index_template/{name}' \
 --header "Authorization: $API_KEY"

Create or update an index template Generally available; Added in 7.9.0

PUT /_index_template/{name}

Index templates define settings, mappings, and aliases that can be applied automatically to new indices.

Elasticsearch applies templates to new indices based on an wildcard pattern that matches the index name. Index templates are applied during data stream or index creation. For data streams, these settings and mappings are applied when the stream's backing indices are created. Settings and mappings specified in a create index API request override any settings or mappings specified in an index template. Changes to index templates do not affect existing indices, including the existing backing indices of a data stream.

You can use C-style /* *\/ block comments in index templates. You can include comments anywhere in the request body, except before the opening curly bracket.

Multiple matching templates

If multiple index templates match the name of a new index or data stream, the template with the highest priority is used.

Multiple templates with overlapping index patterns at the same priority are not allowed and an error will be thrown when attempting to create a template matching an existing index template at identical priorities.

Composing aliases, mappings, and settings

When multiple component templates are specified in the composed_of field for an index template, they are merged in the order specified, meaning that later component templates override earlier component templates. Any mappings, settings, or aliases from the parent index template are merged in next. Finally, any configuration on the index request itself is merged. Mapping definitions are merged recursively, which means that later mapping components can introduce new field mappings and update the mapping configuration. If a field mapping is already contained in an earlier component, its definition will be completely overwritten by the later one. This recursive merging strategy applies not only to field mappings, but also root options like dynamic_templates and meta. If an earlier component contains a dynamic_templates block, then by default new dynamic_templates entries are appended onto the end. If an entry already exists with the same key, then it is overwritten by the new definition.

Required authorization

  • Cluster privileges: manage_index_templates

Path parameters

  • name string Required

    Index or template name

Query parameters

  • create boolean

    If true, this request cannot replace or update existing index templates.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • cause string

    User defined reason for creating/updating the index template

application/json

Body Required

  • index_patterns string | array[string]
  • composed_of array[string]

    An ordered list of component template names. Component templates are merged in the order specified, meaning that the last component template specified has the highest precedence.

  • template object
    Hide template attributes Show template attributes object
    • aliases object

      Aliases to add. If the index template includes a data_stream object, these are data stream aliases. Otherwise, these are index aliases. Data stream aliases ignore the index_routing, routing, and search_routing options.

      Hide aliases attribute Show aliases attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • filter object

          An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

          External documentation
        • index_routing string
        • is_hidden boolean

          If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

        • is_write_index boolean

          If true, the index is the write index for the alias.

        • routing string
        • search_routing string
    • mappings object
      Hide mappings attributes Show mappings attributes object
      • all_field object
        Hide all_field attributes Show all_field attributes object
        • analyzer string Required
        • enabled boolean Required
        • omit_norms boolean Required
        • search_analyzer string Required
        • similarity string Required
        • store boolean Required
        • store_term_vector_offsets boolean Required
        • store_term_vector_payloads boolean Required
        • store_term_vector_positions boolean Required
        • store_term_vectors boolean Required
      • date_detection boolean
      • dynamic string

        Values are strict, runtime, true, or false.

      • dynamic_date_formats array[string]
      • dynamic_templates array[object]
      • _field_names object
        Hide _field_names attribute Show _field_names attribute object
        • enabled boolean Required
      • index_field object
        Hide index_field attribute Show index_field attribute object
        • enabled boolean Required
      • _meta object
        Hide _meta attribute Show _meta attribute object
        • * object Additional properties
      • numeric_detection boolean
      • properties object
      • _routing object
        Hide _routing attribute Show _routing attribute object
        • required boolean Required
      • _size object
        Hide _size attribute Show _size attribute object
        • enabled boolean Required
      • _source object
        Hide _source attributes Show _source attributes object
        • compress boolean
        • compress_threshold string
        • enabled boolean
        • excludes array[string]
        • includes array[string]
        • mode string

          Values are disabled, stored, or synthetic.

      • runtime object
        Hide runtime attribute Show runtime attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • fields object

            For type composite

            Hide fields attribute Show fields attribute object
            • * object Additional properties
              Hide * attribute Show * attribute object
              • type string Required

                Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

          • fetch_fields array[object]

            For type lookup

            Hide fetch_fields attributes Show fetch_fields attributes object
            • field string Required

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • format string
          • format string

            A custom format for date type runtime fields.

          • input_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_index string
          • script object
            Hide script attributes Show script attributes object
            • source string

              The script source.

            • id string
            • params object

              Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

              Hide params attribute Show params attribute object
              • * object Additional properties
            • lang string

              Any of:

              Values are painless, expression, mustache, or java.

            • options object
              Hide options attribute Show options attribute object
              • * string Additional properties
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • enabled boolean
      • subobjects string

        Values are true or false.

      • _data_stream_timestamp object
        Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
        • enabled boolean Required
    • settings object
      Index settings
    • lifecycle object

      Data stream lifecycle denotes that a data stream is managed by the data stream lifecycle and contains the configuration.

      Hide lifecycle attributes Show lifecycle attributes object
      • data_retention string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • downsampling object
        Hide downsampling attribute Show downsampling attribute object
        • rounds array[object] Required

          The list of downsampling rounds to execute as part of this downsampling configuration

          Hide rounds attributes Show rounds attributes object
          • after string Required

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • config object Required
            Hide config attribute Show config attribute object
            • fixed_interval string Required

              A date histogram interval. Similar to Duration with additional units: w (week), M (month), q (quarter) and y (year)

      • enabled boolean

        If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

  • data_stream object
    Hide data_stream attributes Show data_stream attributes object
    • hidden boolean
    • allow_custom_routing boolean
  • priority number

    Priority to determine index template precedence when a new data stream or index is created. The index template with the highest priority is chosen. If no priority is specified the template is treated as though it is of priority 0 (lowest priority). This number is not automatically generated by Elasticsearch.

  • version number
  • _meta object
    Hide _meta attribute Show _meta attribute object
    • * object Additional properties
  • allow_auto_create boolean

    This setting overrides the value of the action.auto_create_index cluster setting. If set to true in a template, then indices can be automatically created using that template even if auto-creation of indices is disabled via actions.auto_create_index. If set to false, then indices or data streams matching the template must always be explicitly created, and may never be automatically created.

  • ignore_missing_component_templates array[string]

    The configuration option ignore_missing_component_templates can be used when an index template references a component template that might not exist

  • deprecated boolean

    Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_index_template/{name}
curl \
 --request PUT 'http://api.example.com/_index_template/{name}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"index_patterns\" : [\"template*\"],\n  \"priority\" : 1,\n  \"template\": {\n    \"settings\" : {\n      \"number_of_shards\" : 2\n    }\n  }\n}"'
Request examples
{
  "index_patterns" : ["template*"],
  "priority" : 1,
  "template": {
    "settings" : {
      "number_of_shards" : 2
    }
  }
}
You can include index aliases in an index template. During index creation, the `{index}` placeholder in the alias name will be replaced with the actual index name that the template gets applied to.
{
  "index_patterns": [
    "template*"
  ],
  "template": {
    "settings": {
      "number_of_shards": 1
    },
    "aliases": {
      "alias1": {},
      "alias2": {
        "filter": {
          "term": {
            "user.id": "kimchy"
          }
        },
        "routing": "shard-1"
      },
      "{index}-alias": {}
    }
  }
}




Delete an index template Generally available; Added in 7.8.0

DELETE /_index_template/{name}

The provided may contain multiple template names separated by a comma. If multiple template names are specified then there is no wildcard support and the provided names should match completely with existing templates.

Required authorization

  • Cluster privileges: manage_index_templates

Path parameters

  • name string | array[string] Required

    Comma-separated list of index template names used to limit the request. Wildcard (*) expressions are supported.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

DELETE /_index_template/{name}
curl \
 --request DELETE 'http://api.example.com/_index_template/{name}' \
 --header "Authorization: $API_KEY"




Get legacy index templates Deprecated Generally available

GET /_template/{name}

Get information about one or more index templates.

IMPORTANT: This documentation is about legacy index templates, which are deprecated and will be replaced by the composable templates introduced in Elasticsearch 7.8.

Required authorization

  • Cluster privileges: manage_index_templates
External documentation

Path parameters

  • name string | array[string] Required

    Comma-separated list of index template names used to limit the request. Wildcard (*) expressions are supported. To return all index templates, omit this parameter or use a value of _all or *.

Query parameters

  • flat_settings boolean

    If true, returns settings in flat format.

  • local boolean

    If true, the request retrieves information from the local node only.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • aliases object Required
        Hide aliases attribute Show aliases attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • filter object

            An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

            External documentation
          • index_routing string
          • is_hidden boolean

            If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

          • is_write_index boolean

            If true, the index is the write index for the alias.

          • routing string
          • search_routing string
      • index_patterns array[string] Required
      • mappings object Required
        Hide mappings attributes Show mappings attributes object
        • all_field object
          Hide all_field attributes Show all_field attributes object
          • analyzer string Required
          • enabled boolean Required
          • omit_norms boolean Required
          • search_analyzer string Required
          • similarity string Required
          • store boolean Required
          • store_term_vector_offsets boolean Required
          • store_term_vector_payloads boolean Required
          • store_term_vector_positions boolean Required
          • store_term_vectors boolean Required
        • date_detection boolean
        • dynamic string

          Values are strict, runtime, true, or false.

        • dynamic_date_formats array[string]
        • dynamic_templates array[object]
        • _field_names object
          Hide _field_names attribute Show _field_names attribute object
          • enabled boolean Required
        • index_field object
          Hide index_field attribute Show index_field attribute object
          • enabled boolean Required
        • _meta object
          Hide _meta attribute Show _meta attribute object
          • * object Additional properties
        • numeric_detection boolean
        • properties object
        • _routing object
          Hide _routing attribute Show _routing attribute object
          • required boolean Required
        • _size object
          Hide _size attribute Show _size attribute object
          • enabled boolean Required
        • _source object
          Hide _source attributes Show _source attributes object
          • compress boolean
          • compress_threshold string
          • enabled boolean
          • excludes array[string]
          • includes array[string]
          • mode string

            Values are disabled, stored, or synthetic.

        • runtime object
          Hide runtime attribute Show runtime attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • fields object

              For type composite

              Hide fields attribute Show fields attribute object
              • * object Additional properties
            • fetch_fields array[object]

              For type lookup

            • format string

              A custom format for date type runtime fields.

            • input_field string

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • target_field string

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • target_index string
            • script object
              Hide script attributes Show script attributes object
              • source string

                The script source.

              • id string
              • params object

                Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

              • lang
              • options object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • enabled boolean
        • subobjects string

          Values are true or false.

        • _data_stream_timestamp object
          Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
          • enabled boolean Required
      • order number Required
      • settings object Required
        Hide settings attribute Show settings attribute object
        • * object Additional properties
      • version number
GET /_template/{name}
curl \
 --request GET 'http://api.example.com/_template/{name}' \
 --header "Authorization: $API_KEY"

Create or update a legacy index template Deprecated Generally available

PUT /_template/{name}

Index templates define settings, mappings, and aliases that can be applied automatically to new indices. Elasticsearch applies templates to new indices based on an index pattern that matches the index name.

IMPORTANT: This documentation is about legacy index templates, which are deprecated and will be replaced by the composable templates introduced in Elasticsearch 7.8.

Composable templates always take precedence over legacy templates. If no composable template matches a new index, matching legacy templates are applied according to their order.

Index templates are only applied during index creation. Changes to index templates do not affect existing indices. Settings and mappings specified in create index API requests override any settings or mappings specified in an index template.

You can use C-style /* *\/ block comments in index templates. You can include comments anywhere in the request body, except before the opening curly bracket.

Indices matching multiple templates

Multiple index templates can potentially match an index, in this case, both the settings and mappings are merged into the final configuration of the index. The order of the merging can be controlled using the order parameter, with lower order being applied first, and higher orders overriding them. NOTE: Multiple matching templates with the same order value will result in a non-deterministic merging order.

Required authorization

  • Cluster privileges: manage_index_templates,manage
External documentation

Path parameters

  • name string Required

    The name of the template

Query parameters

  • create boolean

    If true, this request cannot replace or update existing index templates.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • order number

    Order in which Elasticsearch applies this template if index matches multiple templates.

    Templates with lower 'order' values are merged first. Templates with higher 'order' values are merged later, overriding templates with lower values.

  • cause string

    User defined reason for creating/updating the index template

application/json

Body Required

  • aliases object

    Aliases for the index.

    Hide aliases attribute Show aliases attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • filter object

        An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

        External documentation
      • index_routing string
      • is_hidden boolean

        If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

      • is_write_index boolean

        If true, the index is the write index for the alias.

      • routing string
      • search_routing string
  • index_patterns string | array[string]

    Array of wildcard expressions used to match the names of indices during creation.

  • mappings object
    Hide mappings attributes Show mappings attributes object
    • all_field object
      Hide all_field attributes Show all_field attributes object
      • analyzer string Required
      • enabled boolean Required
      • omit_norms boolean Required
      • search_analyzer string Required
      • similarity string Required
      • store boolean Required
      • store_term_vector_offsets boolean Required
      • store_term_vector_payloads boolean Required
      • store_term_vector_positions boolean Required
      • store_term_vectors boolean Required
    • date_detection boolean
    • dynamic string

      Values are strict, runtime, true, or false.

    • dynamic_date_formats array[string]
    • dynamic_templates array[object]
    • _field_names object
      Hide _field_names attribute Show _field_names attribute object
      • enabled boolean Required
    • index_field object
      Hide index_field attribute Show index_field attribute object
      • enabled boolean Required
    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties
    • numeric_detection boolean
    • properties object
    • _routing object
      Hide _routing attribute Show _routing attribute object
      • required boolean Required
    • _size object
      Hide _size attribute Show _size attribute object
      • enabled boolean Required
    • _source object
      Hide _source attributes Show _source attributes object
      • compress boolean
      • compress_threshold string
      • enabled boolean
      • excludes array[string]
      • includes array[string]
      • mode string

        Values are disabled, stored, or synthetic.

    • runtime object
      Hide runtime attribute Show runtime attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • format string
        • format string

          A custom format for date type runtime fields.

        • input_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_index string
        • script object
          Hide script attributes Show script attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties
          • lang string

            Any of:

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • type string Required

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • enabled boolean
    • subobjects string

      Values are true or false.

    • _data_stream_timestamp object
      Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
      • enabled boolean Required
  • order number

    Order in which Elasticsearch applies this template if index matches multiple templates.

    Templates with lower 'order' values are merged first. Templates with higher 'order' values are merged later, overriding templates with lower values.

  • settings object
    Index settings
  • version number

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_template/{name}
curl \
 --request PUT 'http://api.example.com/_template/{name}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"index_patterns\": [\n    \"te*\",\n    \"bar*\"\n  ],\n  \"settings\": {\n    \"number_of_shards\": 1\n  },\n  \"mappings\": {\n    \"_source\": {\n      \"enabled\": false\n    }\n  },\n  \"properties\": {\n    \"host_name\": {\n      \"type\": \"keyword\"\n    },\n    \"created_at\": {\n      \"type\": \"date\",\n      \"format\": \"EEE MMM dd HH:mm:ss Z yyyy\"\n    }\n  }\n}"'
{
  "index_patterns": [
    "te*",
    "bar*"
  ],
  "settings": {
    "number_of_shards": 1
  },
  "mappings": {
    "_source": {
      "enabled": false
    }
  },
  "properties": {
    "host_name": {
      "type": "keyword"
    },
    "created_at": {
      "type": "date",
      "format": "EEE MMM dd HH:mm:ss Z yyyy"
    }
  }
}
You can include index aliases in an index template. During index creation, the `{index}` placeholder in the alias name will be replaced with the actual index name that the template gets applied to.
{
  "index_patterns": [
    "te*"
  ],
  "settings": {
    "number_of_shards": 1
  },
  "aliases": {
    "alias1": {},
    "alias2": {
      "filter": {
        "term": {
          "user.id": "kimchy"
        }
      },
      "routing": "shard-1"
    },
    "{index}-alias": {}
  }
}

Create or update a legacy index template Deprecated Generally available

POST /_template/{name}

Index templates define settings, mappings, and aliases that can be applied automatically to new indices. Elasticsearch applies templates to new indices based on an index pattern that matches the index name.

IMPORTANT: This documentation is about legacy index templates, which are deprecated and will be replaced by the composable templates introduced in Elasticsearch 7.8.

Composable templates always take precedence over legacy templates. If no composable template matches a new index, matching legacy templates are applied according to their order.

Index templates are only applied during index creation. Changes to index templates do not affect existing indices. Settings and mappings specified in create index API requests override any settings or mappings specified in an index template.

You can use C-style /* *\/ block comments in index templates. You can include comments anywhere in the request body, except before the opening curly bracket.

Indices matching multiple templates

Multiple index templates can potentially match an index, in this case, both the settings and mappings are merged into the final configuration of the index. The order of the merging can be controlled using the order parameter, with lower order being applied first, and higher orders overriding them. NOTE: Multiple matching templates with the same order value will result in a non-deterministic merging order.

Required authorization

  • Cluster privileges: manage_index_templates,manage
External documentation

Path parameters

  • name string Required

    The name of the template

Query parameters

  • create boolean

    If true, this request cannot replace or update existing index templates.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • order number

    Order in which Elasticsearch applies this template if index matches multiple templates.

    Templates with lower 'order' values are merged first. Templates with higher 'order' values are merged later, overriding templates with lower values.

  • cause string

    User defined reason for creating/updating the index template

application/json

Body Required

  • aliases object

    Aliases for the index.

    Hide aliases attribute Show aliases attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • filter object

        An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

        External documentation
      • index_routing string
      • is_hidden boolean

        If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

      • is_write_index boolean

        If true, the index is the write index for the alias.

      • routing string
      • search_routing string
  • index_patterns string | array[string]

    Array of wildcard expressions used to match the names of indices during creation.

  • mappings object
    Hide mappings attributes Show mappings attributes object
    • all_field object
      Hide all_field attributes Show all_field attributes object
      • analyzer string Required
      • enabled boolean Required
      • omit_norms boolean Required
      • search_analyzer string Required
      • similarity string Required
      • store boolean Required
      • store_term_vector_offsets boolean Required
      • store_term_vector_payloads boolean Required
      • store_term_vector_positions boolean Required
      • store_term_vectors boolean Required
    • date_detection boolean
    • dynamic string

      Values are strict, runtime, true, or false.

    • dynamic_date_formats array[string]
    • dynamic_templates array[object]
    • _field_names object
      Hide _field_names attribute Show _field_names attribute object
      • enabled boolean Required
    • index_field object
      Hide index_field attribute Show index_field attribute object
      • enabled boolean Required
    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties
    • numeric_detection boolean
    • properties object
    • _routing object
      Hide _routing attribute Show _routing attribute object
      • required boolean Required
    • _size object
      Hide _size attribute Show _size attribute object
      • enabled boolean Required
    • _source object
      Hide _source attributes Show _source attributes object
      • compress boolean
      • compress_threshold string
      • enabled boolean
      • excludes array[string]
      • includes array[string]
      • mode string

        Values are disabled, stored, or synthetic.

    • runtime object
      Hide runtime attribute Show runtime attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • format string
        • format string

          A custom format for date type runtime fields.

        • input_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_index string
        • script object
          Hide script attributes Show script attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties
          • lang string

            Any of:

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • type string Required

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • enabled boolean
    • subobjects string

      Values are true or false.

    • _data_stream_timestamp object
      Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
      • enabled boolean Required
  • order number

    Order in which Elasticsearch applies this template if index matches multiple templates.

    Templates with lower 'order' values are merged first. Templates with higher 'order' values are merged later, overriding templates with lower values.

  • settings object
    Index settings
  • version number

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /_template/{name}
curl \
 --request POST 'http://api.example.com/_template/{name}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"index_patterns\": [\n    \"te*\",\n    \"bar*\"\n  ],\n  \"settings\": {\n    \"number_of_shards\": 1\n  },\n  \"mappings\": {\n    \"_source\": {\n      \"enabled\": false\n    }\n  },\n  \"properties\": {\n    \"host_name\": {\n      \"type\": \"keyword\"\n    },\n    \"created_at\": {\n      \"type\": \"date\",\n      \"format\": \"EEE MMM dd HH:mm:ss Z yyyy\"\n    }\n  }\n}"'
{
  "index_patterns": [
    "te*",
    "bar*"
  ],
  "settings": {
    "number_of_shards": 1
  },
  "mappings": {
    "_source": {
      "enabled": false
    }
  },
  "properties": {
    "host_name": {
      "type": "keyword"
    },
    "created_at": {
      "type": "date",
      "format": "EEE MMM dd HH:mm:ss Z yyyy"
    }
  }
}
You can include index aliases in an index template. During index creation, the `{index}` placeholder in the alias name will be replaced with the actual index name that the template gets applied to.
{
  "index_patterns": [
    "te*"
  ],
  "settings": {
    "number_of_shards": 1
  },
  "aliases": {
    "alias1": {},
    "alias2": {
      "filter": {
        "term": {
          "user.id": "kimchy"
        }
      },
      "routing": "shard-1"
    },
    "{index}-alias": {}
  }
}








Analyze the index disk usage Technical preview; Added in 7.15.0

POST /{index}/_disk_usage

Analyze the disk usage of each field of an index or data stream. This API might not support indices created in previous Elasticsearch versions. The result of a small index can be inaccurate as some parts of an index might not be analyzed by the API.

NOTE: The total size of fields of the analyzed shards of the index in the response is usually smaller than the index store_size value because some small metadata files are ignored and some parts of data files might not be scanned by the API. Since stored fields are stored together in a compressed format, the sizes of stored fields are also estimates and can be inaccurate. The stored size of the _id field is likely underestimated while the _source field is overestimated.

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases used to limit the request. It’s recommended to execute this API with a single index (or the latest backing index of a data stream) as the API consumes resources significantly.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • flush boolean

    If true, the API performs a flush before analysis. If false, the response may not include uncommitted data.

  • ignore_unavailable boolean

    If true, missing or closed indices are not included in the response.

  • run_expensive_tasks boolean

    Analyzing field disk usage is resource-intensive. To use the API, this parameter must be set to true.

Responses

  • 200 application/json
POST /{index}/_disk_usage
curl \
 --request POST 'http://api.example.com/{index}/_disk_usage' \
 --header "Authorization: $API_KEY"








Get field usage stats Technical preview; Added in 7.15.0

GET /{index}/_field_usage_stats

Get field usage information for each shard and field of an index. Field usage statistics are automatically captured when queries are running on a cluster. A shard-level search request that accesses a given field, even if multiple times during that request, is counted as a single use.

The response body reports the per-shard usage count of the data structures that back the fields in the index. A given request will increment each count by a maximum value of 1, even if the request accesses the same field multiple times.

Required authorization

  • Index privileges: manage

Path parameters

  • index string | array[string] Required

    Comma-separated list or wildcard expression of index names used to limit the request.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If true, missing or closed indices are not included in the response.

  • fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in the statistics.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • _shards object Required
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number Required
        • status string
      • skipped number
GET /{index}/_field_usage_stats
curl \
 --request GET 'http://api.example.com/{index}/_field_usage_stats' \
 --header "Authorization: $API_KEY"
Response examples (200)
An abbreviated response from `GET /my-index-000001/_field_usage_stats`. The `all_fields` object reports the sums of the usage counts for all fields in the index (on the listed shard).
{
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "my-index-000001": {
    "shards": [
      {
        "tracking_id": "MpOl0QlTQ4SYYhEe6KgJoQ",
        "tracking_started_at_millis": 1625558985010,
        "routing": {
          "state": "STARTED",
          "primary": true,
          "node": "gA6KeeVzQkGURFCUyV-e8Q",
          "relocating_node": null
        },
        "stats": {
          "all_fields": {
            "any": "6",
            "inverted_index": {
              "terms": 1,
              "postings": 1,
              "proximity": 1,
              "positions": 0,
              "term_frequencies": 1,
              "offsets": 0,
              "payloads": 0
            },
            "stored_fields": 2,
            "doc_values": 1,
            "points": 0,
            "norms": 1,
            "term_vectors": 0,
            "knn_vectors": 0
          },
          "fields": {
            "_id": {
              "any": 1,
              "inverted_index": {
                "terms": 1,
                "postings": 1,
                "proximity": 1,
                "positions": 0,
                "term_frequencies": 1,
                "offsets": 0,
                "payloads": 0
              },
              "stored_fields": 1,
              "doc_values": 0,
              "points": 0,
              "norms": 0,
              "term_vectors": 0,
              "knn_vectors": 0
            },
            "_source": {},
            "context": {},
            "message.keyword": {}
          }
        }
      }
    ]
  }
}

Flush data streams or indices Generally available

GET /_flush

Flushing a data stream or index is the process of making sure that any data that is currently only stored in the transaction log is also permanently stored in the Lucene index. When restarting, Elasticsearch replays any unflushed operations from the transaction log into the Lucene index to bring it back into the state that it was in before the restart. Elasticsearch automatically triggers flushes as needed, using heuristics that trade off the size of the unflushed transaction log against the cost of performing each flush.

After each operation has been flushed it is permanently stored in the Lucene index. This may mean that there is no need to maintain an additional copy of it in the transaction log. The transaction log is made up of multiple files, called generations, and Elasticsearch will delete any generation files when they are no longer needed, freeing up disk space.

It is also possible to trigger a flush on one or more indices using the flush API, although it is rare for users to need to call this API directly. If you call the flush API after indexing some documents then a successful response indicates that Elasticsearch has flushed all the documents that were indexed before the flush API was called.

Required authorization

  • Index privileges: maintenance

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • force boolean

    If true, the request forces a flush even if there are no changes to commit to the index.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • wait_if_ongoing boolean

    If true, the flush operation blocks until execution when another flush operation is running. If false, Elasticsearch returns an error if you request a flush when another flush operation is running.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • _shards object
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number Required
        • status string
      • skipped number
GET /_flush
curl \
 --request GET 'http://api.example.com/_flush' \
 --header "Authorization: $API_KEY"

Flush data streams or indices Generally available

POST /_flush

Flushing a data stream or index is the process of making sure that any data that is currently only stored in the transaction log is also permanently stored in the Lucene index. When restarting, Elasticsearch replays any unflushed operations from the transaction log into the Lucene index to bring it back into the state that it was in before the restart. Elasticsearch automatically triggers flushes as needed, using heuristics that trade off the size of the unflushed transaction log against the cost of performing each flush.

After each operation has been flushed it is permanently stored in the Lucene index. This may mean that there is no need to maintain an additional copy of it in the transaction log. The transaction log is made up of multiple files, called generations, and Elasticsearch will delete any generation files when they are no longer needed, freeing up disk space.

It is also possible to trigger a flush on one or more indices using the flush API, although it is rare for users to need to call this API directly. If you call the flush API after indexing some documents then a successful response indicates that Elasticsearch has flushed all the documents that were indexed before the flush API was called.

Required authorization

  • Index privileges: maintenance

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • force boolean

    If true, the request forces a flush even if there are no changes to commit to the index.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • wait_if_ongoing boolean

    If true, the flush operation blocks until execution when another flush operation is running. If false, Elasticsearch returns an error if you request a flush when another flush operation is running.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • _shards object
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number Required
        • status string
      • skipped number
POST /_flush
curl \
 --request POST 'http://api.example.com/_flush' \
 --header "Authorization: $API_KEY"

Flush data streams or indices Generally available

GET /{index}/_flush

Flushing a data stream or index is the process of making sure that any data that is currently only stored in the transaction log is also permanently stored in the Lucene index. When restarting, Elasticsearch replays any unflushed operations from the transaction log into the Lucene index to bring it back into the state that it was in before the restart. Elasticsearch automatically triggers flushes as needed, using heuristics that trade off the size of the unflushed transaction log against the cost of performing each flush.

After each operation has been flushed it is permanently stored in the Lucene index. This may mean that there is no need to maintain an additional copy of it in the transaction log. The transaction log is made up of multiple files, called generations, and Elasticsearch will delete any generation files when they are no longer needed, freeing up disk space.

It is also possible to trigger a flush on one or more indices using the flush API, although it is rare for users to need to call this API directly. If you call the flush API after indexing some documents then a successful response indicates that Elasticsearch has flushed all the documents that were indexed before the flush API was called.

Required authorization

  • Index privileges: maintenance

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases to flush. Supports wildcards (*). To flush all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • force boolean

    If true, the request forces a flush even if there are no changes to commit to the index.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • wait_if_ongoing boolean

    If true, the flush operation blocks until execution when another flush operation is running. If false, Elasticsearch returns an error if you request a flush when another flush operation is running.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • _shards object
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number Required
        • status string
      • skipped number
GET /{index}/_flush
curl \
 --request GET 'http://api.example.com/{index}/_flush' \
 --header "Authorization: $API_KEY"




Force a merge Generally available; Added in 2.1.0

POST /_forcemerge

Perform the force merge operation on the shards of one or more indices. For data streams, the API forces a merge on the shards of the stream's backing indices.

Merging reduces the number of segments in each shard by merging some of them together and also frees up the space used by deleted documents. Merging normally happens automatically, but sometimes it is useful to trigger a merge manually.

WARNING: We recommend force merging only a read-only index (meaning the index is no longer receiving writes). When documents are updated or deleted, the old version is not immediately removed but instead soft-deleted and marked with a "tombstone". These soft-deleted documents are automatically cleaned up during regular segment merges. But force merge can cause very large (greater than 5 GB) segments to be produced, which are not eligible for regular merges. So the number of soft-deleted documents can then grow rapidly, resulting in higher disk usage and worse search performance. If you regularly force merge an index receiving writes, this can also make snapshots more expensive, since the new documents can't be backed up incrementally.

Blocks during a force merge

Calls to this API block until the merge is complete (unless request contains wait_for_completion=false). If the client connection is lost before completion then the force merge process will continue in the background. Any new requests to force merge the same indices will also block until the ongoing force merge is complete.

Running force merge asynchronously

If the request contains wait_for_completion=false, Elasticsearch performs some preflight checks, launches the request, and returns a task you can use to get the status of the task. However, you can not cancel this task as the force merge task is not cancelable. Elasticsearch creates a record of this task as a document at _tasks/<task_id>. When you are done with a task, you should delete the task document so Elasticsearch can reclaim the space.

Force merging multiple indices

You can force merge multiple indices with a single request by targeting:

  • One or more data streams that contain multiple backing indices
  • Multiple indices
  • One or more aliases
  • All data streams and indices in a cluster

Each targeted shard is force-merged separately using the force_merge threadpool. By default each node only has a single force_merge thread which means that the shards on that node are force-merged one at a time. If you expand the force_merge threadpool on a node then it will force merge its shards in parallel

Force merge makes the storage for the shard being merged temporarily increase, as it may require free space up to triple its size in case max_num_segments parameter is set to 1, to rewrite all segments into a new one.

Data streams and time-based indices

Force-merging is useful for managing a data stream's older backing indices and other time-based indices, particularly after a rollover. In these cases, each index only receives indexing traffic for a certain period of time. Once an index receive no more writes, its shards can be force-merged to a single segment. This can be a good idea because single-segment shards can sometimes use simpler and more efficient data structures to perform searches. For example:

POST /.ds-my-data-stream-2099.03.07-000001/_forcemerge?max_num_segments=1

Required authorization

  • Index privileges: maintenance
External documentation

Query parameters

  • allow_no_indices boolean

    Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)

  • expand_wildcards string | array[string]

    Whether to expand wildcard expression to concrete indices that are open, closed or both.

    Values are all, open, closed, hidden, or none.

  • flush boolean

    Specify whether the index should be flushed after performing the operation (default: true)

  • ignore_unavailable boolean

    Whether specified concrete indices should be ignored when unavailable (missing or closed)

  • max_num_segments number

    The number of segments the index should be merged into (default: dynamic)

  • only_expunge_deletes boolean

    Specify whether the operation should only expunge deleted documents

  • wait_for_completion boolean

    Should the request wait until the force merge is completed.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _shards object
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason
          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]
          • suppressed array[object]
        • shard number Required
        • status string
      • skipped number
    • task string

      task contains a task id returned when wait_for_completion=false, you can use the task_id to get the status of the task at _tasks/

POST /_forcemerge
curl \
 --request POST 'http://api.example.com/_forcemerge' \
 --header "Authorization: $API_KEY"
















Get mapping definitions Generally available

GET /{index}/_mapping/field/{fields}

Retrieves mapping definitions for one or more fields. For data streams, the API retrieves field mappings for the stream’s backing indices.

This API is useful if you don't need a complete mapping or if an index mapping contains a large number of fields.

Required authorization

  • Index privileges: view_index_metadata

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

  • fields string | array[string] Required

    Comma-separated list or wildcard expression of fields used to limit returned information. Supports wildcards (*).

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • include_defaults boolean

    If true, return all default settings in the response.

  • local boolean

    If true, the request retrieves information from the local node only.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attribute Show * attribute object
      • mappings object Required
        Hide mappings attribute Show mappings attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • full_name string Required
          • mapping object Required
GET /{index}/_mapping/field/{fields}
curl \
 --request GET 'http://api.example.com/{index}/_mapping/field/{fields}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A sucessful response from `GET publications/_mapping/field/title`, which returns the mapping of a field called `title`.
{
   "publications": {
      "mappings": {
          "title": {
             "full_name": "title",
             "mapping": {
                "title": {
                   "type": "text"
                }
             }
          }
       }
   }
}
A successful response from `GET publications/_mapping/field/author.id,abstract,name`. The get field mapping API also supports wildcard notation.
{
   "publications": {
      "mappings": {
        "author.id": {
           "full_name": "author.id",
           "mapping": {
              "id": {
                 "type": "text"
              }
           }
        },
        "abstract": {
           "full_name": "abstract",
           "mapping": {
              "abstract": {
                 "type": "text"
              }
           }
        }
     }
   }
}
A successful response from `GET publications/_mapping/field/a*`.
{
   "publications": {
      "mappings": {
         "author.name": {
            "full_name": "author.name",
            "mapping": {
               "name": {
                 "type": "text"
               }
            }
         },
         "abstract": {
            "full_name": "abstract",
            "mapping": {
               "abstract": {
                  "type": "text"
               }
            }
         },
         "author.id": {
            "full_name": "author.id",
            "mapping": {
               "id": {
                  "type": "text"
               }
            }
         }
      }
   }
}

Get index templates Generally available; Added in 7.9.0

GET /_index_template

Get information about one or more index templates.

Required authorization

  • Cluster privileges: manage_index_templates

Query parameters

  • local boolean

    If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.

  • flat_settings boolean

    If true, returns settings in flat format.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • include_defaults boolean

    If true, returns all relevant default configurations for the index template.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • index_templates array[object] Required
      Hide index_templates attributes Show index_templates attributes object
      • name string Required
      • index_template object Required
        Hide index_template attributes Show index_template attributes object
        • index_patterns string | array[string] Required
        • composed_of array[string] Required

          An ordered list of component template names. Component templates are merged in the order specified, meaning that the last component template specified has the highest precedence.

        • template object
          Hide template attributes Show template attributes object
          • aliases object

            Aliases to add. If the index template includes a data_stream object, these are data stream aliases. Otherwise, these are index aliases. Data stream aliases ignore the index_routing, routing, and search_routing options.

            Hide aliases attribute Show aliases attribute object
            • * object Additional properties
          • mappings object
            Hide mappings attributes Show mappings attributes object
            • all_field object
            • date_detection boolean
            • dynamic string

              Values are strict, runtime, true, or false.

            • dynamic_date_formats array[string]
            • dynamic_templates array[object]
            • _field_names object
            • index_field object
            • _meta object
            • numeric_detection boolean
            • properties object
            • _routing object
            • _size object
            • _source object
            • runtime object
            • enabled boolean
            • subobjects string

              Values are true or false.

            • _data_stream_timestamp object
          • settings object
            Index settings
          • lifecycle object
        • version number
        • priority number

          Priority to determine index template precedence when a new data stream or index is created. The index template with the highest priority is chosen. If no priority is specified the template is treated as though it is of priority 0 (lowest priority). This number is not automatically generated by Elasticsearch.

        • _meta object
          Hide _meta attribute Show _meta attribute object
          • * object Additional properties
        • allow_auto_create boolean
        • data_stream object
          Hide data_stream attributes Show data_stream attributes object
          • hidden boolean

            If true, the data stream is hidden.

          • allow_custom_routing boolean

            If true, the data stream supports custom routing.

        • deprecated boolean Generally available; Added in 8.12.0

          Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.

        • ignore_missing_component_templates string | array[string]
GET /_index_template
curl \
 --request GET 'http://api.example.com/_index_template' \
 --header "Authorization: $API_KEY"








Update field mappings Generally available

PUT /{index}/_mapping

Add new fields to an existing data stream or index. You can also use this API to change the search settings of existing fields and add new properties to existing object fields. For data streams, these changes are applied to all backing indices by default.

Add multi-fields to an existing field

Multi-fields let you index the same field in different ways. You can use this API to update the fields mapping parameter and enable multi-fields for an existing field. WARNING: If an index (or data stream) contains documents when you add a multi-field, those documents will not have values for the new multi-field. You can populate the new multi-field with the update by query API.

Change supported mapping parameters for an existing field

The documentation for each mapping parameter indicates whether you can update it for an existing field using this API. For example, you can use the update mapping API to update the ignore_above parameter.

Change the mapping of an existing field

Except for supported mapping parameters, you can't change the mapping or field type of an existing field. Changing an existing field could invalidate data that's already indexed.

If you need to change the mapping of a field in a data stream's backing indices, refer to documentation about modifying data streams. If you need to change the mapping of a field in other indices, create a new index with the correct mapping and reindex your data into that index.

Rename a field

Renaming a field would invalidate data already indexed under the old field name. Instead, add an alias field to create an alternate field name.

Required authorization

  • Index privileges: manage
External documentation

Path parameters

  • index string | array[string] Required

    A comma-separated list of index names the mapping should be added to (supports wildcards); use _all or omit to add the mapping on all indices.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • write_index_only boolean

    If true, the mappings are applied only to the current write index for the target.

application/json

Body Required

  • date_detection boolean

    Controls whether dynamic date detection is enabled.

  • dynamic string

    Values are strict, runtime, true, or false.

  • dynamic_date_formats array[string]

    If date detection is enabled then new string fields are checked against 'dynamic_date_formats' and if the value matches then a new date field is added instead of string.

  • dynamic_templates array[object]

    Specify dynamic templates for the mapping.

  • _field_names object
    Hide _field_names attribute Show _field_names attribute object
    • enabled boolean Required
  • _meta object
    Hide _meta attribute Show _meta attribute object
    • * object Additional properties
  • numeric_detection boolean

    Automatically map strings into numeric data types for all fields.

  • properties object

    Mapping for a field. For new fields, this mapping can include:

    • Field name
    • Field data type
    • Mapping parameters
  • _routing object
    Hide _routing attribute Show _routing attribute object
    • required boolean Required
  • _source object
    Hide _source attributes Show _source attributes object
    • compress boolean
    • compress_threshold string
    • enabled boolean
    • excludes array[string]
    • includes array[string]
    • mode string

      Values are disabled, stored, or synthetic.

  • runtime object
    Hide runtime attribute Show runtime attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • fields object

        For type composite

        Hide fields attribute Show fields attribute object
        • * object Additional properties
          Hide * attribute Show * attribute object
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • fetch_fields array[object]

        For type lookup

        Hide fetch_fields attributes Show fetch_fields attributes object
        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • format string
      • format string

        A custom format for date type runtime fields.

      • input_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_index string
      • script object
        Hide script attributes Show script attributes object
        • source string

          The script source.

        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          Hide params attribute Show params attribute object
          • * object Additional properties
        • lang string

          Any of:

          Values are painless, expression, mustache, or java.

        • options object
          Hide options attribute Show options attribute object
          • * string Additional properties
      • type string Required

        Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

    • _shards object
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason
          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]
          • suppressed array[object]
        • shard number Required
        • status string
      • skipped number
PUT /{index}/_mapping
curl \
 --request PUT 'http://api.example.com/{index}/_mapping' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"properties\": {\n    \"user\": {\n      \"properties\": {\n        \"name\": {\n          \"type\": \"keyword\"\n        }\n      }\n    }\n  }\n}"'
Request example
The update mapping API can be applied to multiple data streams or indices with a single request. For example, run `PUT /my-index-000001,my-index-000002/_mapping` to update mappings for the `my-index-000001` and `my-index-000002` indices at the same time.
{
  "properties": {
    "user": {
      "properties": {
        "name": {
          "type": "keyword"
        }
      }
    }
  }
}

Update field mappings Generally available

POST /{index}/_mapping

Add new fields to an existing data stream or index. You can also use this API to change the search settings of existing fields and add new properties to existing object fields. For data streams, these changes are applied to all backing indices by default.

Add multi-fields to an existing field

Multi-fields let you index the same field in different ways. You can use this API to update the fields mapping parameter and enable multi-fields for an existing field. WARNING: If an index (or data stream) contains documents when you add a multi-field, those documents will not have values for the new multi-field. You can populate the new multi-field with the update by query API.

Change supported mapping parameters for an existing field

The documentation for each mapping parameter indicates whether you can update it for an existing field using this API. For example, you can use the update mapping API to update the ignore_above parameter.

Change the mapping of an existing field

Except for supported mapping parameters, you can't change the mapping or field type of an existing field. Changing an existing field could invalidate data that's already indexed.

If you need to change the mapping of a field in a data stream's backing indices, refer to documentation about modifying data streams. If you need to change the mapping of a field in other indices, create a new index with the correct mapping and reindex your data into that index.

Rename a field

Renaming a field would invalidate data already indexed under the old field name. Instead, add an alias field to create an alternate field name.

Required authorization

  • Index privileges: manage
External documentation

Path parameters

  • index string | array[string] Required

    A comma-separated list of index names the mapping should be added to (supports wildcards); use _all or omit to add the mapping on all indices.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • write_index_only boolean

    If true, the mappings are applied only to the current write index for the target.

application/json

Body Required

  • date_detection boolean

    Controls whether dynamic date detection is enabled.

  • dynamic string

    Values are strict, runtime, true, or false.

  • dynamic_date_formats array[string]

    If date detection is enabled then new string fields are checked against 'dynamic_date_formats' and if the value matches then a new date field is added instead of string.

  • dynamic_templates array[object]

    Specify dynamic templates for the mapping.

  • _field_names object
    Hide _field_names attribute Show _field_names attribute object
    • enabled boolean Required
  • _meta object
    Hide _meta attribute Show _meta attribute object
    • * object Additional properties
  • numeric_detection boolean

    Automatically map strings into numeric data types for all fields.

  • properties object

    Mapping for a field. For new fields, this mapping can include:

    • Field name
    • Field data type
    • Mapping parameters
  • _routing object
    Hide _routing attribute Show _routing attribute object
    • required boolean Required
  • _source object
    Hide _source attributes Show _source attributes object
    • compress boolean
    • compress_threshold string
    • enabled boolean
    • excludes array[string]
    • includes array[string]
    • mode string

      Values are disabled, stored, or synthetic.

  • runtime object
    Hide runtime attribute Show runtime attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • fields object

        For type composite

        Hide fields attribute Show fields attribute object
        • * object Additional properties
          Hide * attribute Show * attribute object
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • fetch_fields array[object]

        For type lookup

        Hide fetch_fields attributes Show fetch_fields attributes object
        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • format string
      • format string

        A custom format for date type runtime fields.

      • input_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_index string
      • script object
        Hide script attributes Show script attributes object
        • source string

          The script source.

        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          Hide params attribute Show params attribute object
          • * object Additional properties
        • lang string

          Any of:

          Values are painless, expression, mustache, or java.

        • options object
          Hide options attribute Show options attribute object
          • * string Additional properties
      • type string Required

        Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

    • _shards object
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason
          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]
          • suppressed array[object]
        • shard number Required
        • status string
      • skipped number
POST /{index}/_mapping
curl \
 --request POST 'http://api.example.com/{index}/_mapping' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"properties\": {\n    \"user\": {\n      \"properties\": {\n        \"name\": {\n          \"type\": \"keyword\"\n        }\n      }\n    }\n  }\n}"'
Request example
The update mapping API can be applied to multiple data streams or indices with a single request. For example, run `PUT /my-index-000001,my-index-000002/_mapping` to update mappings for the `my-index-000001` and `my-index-000002` indices at the same time.
{
  "properties": {
    "user": {
      "properties": {
        "name": {
          "type": "keyword"
        }
      }
    }
  }
}




Update index settings Generally available

PUT /_settings

Changes dynamic index settings in real time. For data streams, index setting changes are applied to all backing indices by default.

To revert a setting to the default value, use a null value. The list of per-index settings that can be updated dynamically on live indices can be found in index settings documentation. To preserve existing settings from being updated, set the preserve_existing parameter to true.

There are multiple valid ways to represent index settings in the request body. You can specify only the setting, for example:

{
  "number_of_replicas": 1
}

Or you can use an index setting object:

{
  "index": {
    "number_of_replicas": 1
  }
}

Or you can use dot annotation:

{
  "index.number_of_replicas": 1
}

Or you can embed any of the aforementioned options in a settings object. For example:

{
  "settings": {
    "index": {
      "number_of_replicas": 1
    }
  }
}

NOTE: You can only define new analyzers on closed indices. To add an analyzer, you must close the index, define the analyzer, and reopen the index. You cannot close the write index of a data stream. To update the analyzer for a data stream's write index and future backing indices, update the analyzer in the index template used by the stream. Then roll over the data stream to apply the new analyzer to the stream's write index and future backing indices. This affects searches and any new data added to the stream after the rollover. However, it does not affect the data stream's backing indices or their existing data. To change the analyzer for existing backing indices, you must create a new data stream and reindex your data into it.

Required authorization

  • Index privileges: manage
External documentation

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • flat_settings boolean

    If true, returns settings in flat format.

  • ignore_unavailable boolean

    If true, returns settings in flat format.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • preserve_existing boolean

    If true, existing index settings remain unchanged.

  • reopen boolean

    Whether to close and reopen the index to apply non-dynamic settings. If set to true the indices to which the settings are being applied will be closed temporarily and then reopened in order to apply the changes.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

application/json

Body Required

object object
Index settings

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_settings
curl \
 --request PUT 'http://api.example.com/_settings' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"index\" : {\n    \"number_of_replicas\" : 2\n  }\n}"'
{
  "index" : {
    "number_of_replicas" : 2
  }
}
To revert a setting to the default value, use `null`.
{
  "index" : {
    "refresh_interval" : null
  }
}
To add an analyzer, you must close the index, define the analyzer, then reopen the index.
{
  "analysis" : {
    "analyzer":{
      "content":{
        "type":"custom",
        "tokenizer":"whitespace"
      }
    }
  }
}

POST /my-index-000001/_open

Get index settings Generally available

GET /{index}/_settings

Get setting information for one or more indices. For data streams, it returns setting information for the stream's backing indices.

Required authorization

  • Index privileges: view_index_metadata

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • flat_settings boolean

    If true, returns settings in flat format.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • include_defaults boolean

    If true, return all default settings in the response.

  • local boolean

    If true, the request retrieves information from the local node only. If false, information is retrieved from the master node.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object
      Hide * attributes Show * attributes object
      • aliases object
        Hide aliases attribute Show aliases attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • filter object

            An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

            External documentation
          • index_routing string
          • is_hidden boolean

            If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

          • is_write_index boolean

            If true, the index is the write index for the alias.

          • routing string
          • search_routing string
      • mappings object
        Hide mappings attributes Show mappings attributes object
        • all_field object
          Hide all_field attributes Show all_field attributes object
          • analyzer string Required
          • enabled boolean Required
          • omit_norms boolean Required
          • search_analyzer string Required
          • similarity string Required
          • store boolean Required
          • store_term_vector_offsets boolean Required
          • store_term_vector_payloads boolean Required
          • store_term_vector_positions boolean Required
          • store_term_vectors boolean Required
        • date_detection boolean
        • dynamic string

          Values are strict, runtime, true, or false.

        • dynamic_date_formats array[string]
        • dynamic_templates array[object]
        • _field_names object
          Hide _field_names attribute Show _field_names attribute object
          • enabled boolean Required
        • index_field object
          Hide index_field attribute Show index_field attribute object
          • enabled boolean Required
        • _meta object
          Hide _meta attribute Show _meta attribute object
          • * object Additional properties
        • numeric_detection boolean
        • properties object
        • _routing object
          Hide _routing attribute Show _routing attribute object
          • required boolean Required
        • _size object
          Hide _size attribute Show _size attribute object
          • enabled boolean Required
        • _source object
          Hide _source attributes Show _source attributes object
          • compress boolean
          • compress_threshold string
          • enabled boolean
          • excludes array[string]
          • includes array[string]
          • mode string

            Values are disabled, stored, or synthetic.

        • runtime object
          Hide runtime attribute Show runtime attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • fields object

              For type composite

              Hide fields attribute Show fields attribute object
              • * object Additional properties
            • fetch_fields array[object]

              For type lookup

            • format string

              A custom format for date type runtime fields.

            • input_field string

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • target_field string

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • target_index string
            • script object
              Hide script attributes Show script attributes object
              • source string

                The script source.

              • id string
              • params object

                Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

              • lang
              • options object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • enabled boolean
        • subobjects string

          Values are true or false.

        • _data_stream_timestamp object
          Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
          • enabled boolean Required
      • settings object
        Index settings
      • defaults object
        Index settings
      • data_stream string
      • lifecycle object

        Data stream lifecycle denotes that a data stream is managed by the data stream lifecycle and contains the configuration.

        Hide lifecycle attributes Show lifecycle attributes object
        • data_retention string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • downsampling object
          Hide downsampling attribute Show downsampling attribute object
          • rounds array[object] Required

            The list of downsampling rounds to execute as part of this downsampling configuration

            Hide rounds attributes Show rounds attributes object
            • after string Required

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • config object Required
        • enabled boolean

          If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

GET /{index}/_settings
curl \
 --request GET 'http://api.example.com/{index}/_settings' \
 --header "Authorization: $API_KEY"

Update index settings Generally available

PUT /{index}/_settings

Changes dynamic index settings in real time. For data streams, index setting changes are applied to all backing indices by default.

To revert a setting to the default value, use a null value. The list of per-index settings that can be updated dynamically on live indices can be found in index settings documentation. To preserve existing settings from being updated, set the preserve_existing parameter to true.

There are multiple valid ways to represent index settings in the request body. You can specify only the setting, for example:

{
  "number_of_replicas": 1
}

Or you can use an index setting object:

{
  "index": {
    "number_of_replicas": 1
  }
}

Or you can use dot annotation:

{
  "index.number_of_replicas": 1
}

Or you can embed any of the aforementioned options in a settings object. For example:

{
  "settings": {
    "index": {
      "number_of_replicas": 1
    }
  }
}

NOTE: You can only define new analyzers on closed indices. To add an analyzer, you must close the index, define the analyzer, and reopen the index. You cannot close the write index of a data stream. To update the analyzer for a data stream's write index and future backing indices, update the analyzer in the index template used by the stream. Then roll over the data stream to apply the new analyzer to the stream's write index and future backing indices. This affects searches and any new data added to the stream after the rollover. However, it does not affect the data stream's backing indices or their existing data. To change the analyzer for existing backing indices, you must create a new data stream and reindex your data into it.

Required authorization

  • Index privileges: manage
External documentation

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • flat_settings boolean

    If true, returns settings in flat format.

  • ignore_unavailable boolean

    If true, returns settings in flat format.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • preserve_existing boolean

    If true, existing index settings remain unchanged.

  • reopen boolean

    Whether to close and reopen the index to apply non-dynamic settings. If set to true the indices to which the settings are being applied will be closed temporarily and then reopened in order to apply the changes.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

application/json

Body Required

object object
Index settings

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /{index}/_settings
curl \
 --request PUT 'http://api.example.com/{index}/_settings' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"index\" : {\n    \"number_of_replicas\" : 2\n  }\n}"'
{
  "index" : {
    "number_of_replicas" : 2
  }
}
To revert a setting to the default value, use `null`.
{
  "index" : {
    "refresh_interval" : null
  }
}
To add an analyzer, you must close the index, define the analyzer, then reopen the index.
{
  "analysis" : {
    "analyzer":{
      "content":{
        "type":"custom",
        "tokenizer":"whitespace"
      }
    }
  }
}

POST /my-index-000001/_open

Get index settings Generally available

GET /{index}/_settings/{name}

Get setting information for one or more indices. For data streams, it returns setting information for the stream's backing indices.

Required authorization

  • Index privileges: view_index_metadata

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

  • name string | array[string] Required

    Comma-separated list or wildcard expression of settings to retrieve.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • flat_settings boolean

    If true, returns settings in flat format.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • include_defaults boolean

    If true, return all default settings in the response.

  • local boolean

    If true, the request retrieves information from the local node only. If false, information is retrieved from the master node.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object
      Hide * attributes Show * attributes object
      • aliases object
        Hide aliases attribute Show aliases attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • filter object

            An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

            External documentation
          • index_routing string
          • is_hidden boolean

            If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

          • is_write_index boolean

            If true, the index is the write index for the alias.

          • routing string
          • search_routing string
      • mappings object
        Hide mappings attributes Show mappings attributes object
        • all_field object
          Hide all_field attributes Show all_field attributes object
          • analyzer string Required
          • enabled boolean Required
          • omit_norms boolean Required
          • search_analyzer string Required
          • similarity string Required
          • store boolean Required
          • store_term_vector_offsets boolean Required
          • store_term_vector_payloads boolean Required
          • store_term_vector_positions boolean Required
          • store_term_vectors boolean Required
        • date_detection boolean
        • dynamic string

          Values are strict, runtime, true, or false.

        • dynamic_date_formats array[string]
        • dynamic_templates array[object]
        • _field_names object
          Hide _field_names attribute Show _field_names attribute object
          • enabled boolean Required
        • index_field object
          Hide index_field attribute Show index_field attribute object
          • enabled boolean Required
        • _meta object
          Hide _meta attribute Show _meta attribute object
          • * object Additional properties
        • numeric_detection boolean
        • properties object
        • _routing object
          Hide _routing attribute Show _routing attribute object
          • required boolean Required
        • _size object
          Hide _size attribute Show _size attribute object
          • enabled boolean Required
        • _source object
          Hide _source attributes Show _source attributes object
          • compress boolean
          • compress_threshold string
          • enabled boolean
          • excludes array[string]
          • includes array[string]
          • mode string

            Values are disabled, stored, or synthetic.

        • runtime object
          Hide runtime attribute Show runtime attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • fields object

              For type composite

              Hide fields attribute Show fields attribute object
              • * object Additional properties
            • fetch_fields array[object]

              For type lookup

            • format string

              A custom format for date type runtime fields.

            • input_field string

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • target_field string

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • target_index string
            • script object
              Hide script attributes Show script attributes object
              • source string

                The script source.

              • id string
              • params object

                Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

              • lang
              • options object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • enabled boolean
        • subobjects string

          Values are true or false.

        • _data_stream_timestamp object
          Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
          • enabled boolean Required
      • settings object
        Index settings
      • defaults object
        Index settings
      • data_stream string
      • lifecycle object

        Data stream lifecycle denotes that a data stream is managed by the data stream lifecycle and contains the configuration.

        Hide lifecycle attributes Show lifecycle attributes object
        • data_retention string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • downsampling object
          Hide downsampling attribute Show downsampling attribute object
          • rounds array[object] Required

            The list of downsampling rounds to execute as part of this downsampling configuration

            Hide rounds attributes Show rounds attributes object
            • after string Required

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • config object Required
        • enabled boolean

          If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

GET /{index}/_settings/{name}
curl \
 --request GET 'http://api.example.com/{index}/_settings/{name}' \
 --header "Authorization: $API_KEY"




Get legacy index templates Deprecated Generally available

GET /_template

Get information about one or more index templates.

IMPORTANT: This documentation is about legacy index templates, which are deprecated and will be replaced by the composable templates introduced in Elasticsearch 7.8.

Required authorization

  • Cluster privileges: manage_index_templates
External documentation

Query parameters

  • flat_settings boolean

    If true, returns settings in flat format.

  • local boolean

    If true, the request retrieves information from the local node only.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • aliases object Required
        Hide aliases attribute Show aliases attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • filter object

            An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

            External documentation
          • index_routing string
          • is_hidden boolean

            If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

          • is_write_index boolean

            If true, the index is the write index for the alias.

          • routing string
          • search_routing string
      • index_patterns array[string] Required
      • mappings object Required
        Hide mappings attributes Show mappings attributes object
        • all_field object
          Hide all_field attributes Show all_field attributes object
          • analyzer string Required
          • enabled boolean Required
          • omit_norms boolean Required
          • search_analyzer string Required
          • similarity string Required
          • store boolean Required
          • store_term_vector_offsets boolean Required
          • store_term_vector_payloads boolean Required
          • store_term_vector_positions boolean Required
          • store_term_vectors boolean Required
        • date_detection boolean
        • dynamic string

          Values are strict, runtime, true, or false.

        • dynamic_date_formats array[string]
        • dynamic_templates array[object]
        • _field_names object
          Hide _field_names attribute Show _field_names attribute object
          • enabled boolean Required
        • index_field object
          Hide index_field attribute Show index_field attribute object
          • enabled boolean Required
        • _meta object
          Hide _meta attribute Show _meta attribute object
          • * object Additional properties
        • numeric_detection boolean
        • properties object
        • _routing object
          Hide _routing attribute Show _routing attribute object
          • required boolean Required
        • _size object
          Hide _size attribute Show _size attribute object
          • enabled boolean Required
        • _source object
          Hide _source attributes Show _source attributes object
          • compress boolean
          • compress_threshold string
          • enabled boolean
          • excludes array[string]
          • includes array[string]
          • mode string

            Values are disabled, stored, or synthetic.

        • runtime object
          Hide runtime attribute Show runtime attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • fields object

              For type composite

              Hide fields attribute Show fields attribute object
              • * object Additional properties
            • fetch_fields array[object]

              For type lookup

            • format string

              A custom format for date type runtime fields.

            • input_field string

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • target_field string

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • target_index string
            • script object
              Hide script attributes Show script attributes object
              • source string

                The script source.

              • id string
              • params object

                Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

              • lang
              • options object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • enabled boolean
        • subobjects string

          Values are true or false.

        • _data_stream_timestamp object
          Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
          • enabled boolean Required
      • order number Required
      • settings object Required
        Hide settings attribute Show settings attribute object
        • * object Additional properties
      • version number
GET /_template
curl \
 --request GET 'http://api.example.com/_template' \
 --header "Authorization: $API_KEY"












Refresh an index Generally available

GET /_refresh

A refresh makes recent operations performed on one or more indices available for search. For data streams, the API runs the refresh operation on the stream’s backing indices.

By default, Elasticsearch periodically refreshes indices every second, but only on indices that have received one search request or more in the last 30 seconds. You can change this default interval with the index.refresh_interval setting.

Refresh requests are synchronous and do not return a response until the refresh operation completes.

Refreshes are resource-intensive. To ensure good cluster performance, it's recommended to wait for Elasticsearch's periodic refresh rather than performing an explicit refresh when possible.

If your application workflow indexes documents and then runs a search to retrieve the indexed document, it's recommended to use the index API's refresh=wait_for query parameter option. This option ensures the indexing operation waits for a periodic refresh before running the search.

Required authorization

  • Index privileges: maintenance

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • _shards object
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number Required
        • status string
      • skipped number
GET /_refresh
curl \
 --request GET 'http://api.example.com/_refresh' \
 --header "Authorization: $API_KEY"
















Reload search analyzers Generally available; Added in 7.3.0

POST /{index}/_reload_search_analyzers

Reload an index's search analyzers and their resources. For data streams, the API reloads search analyzers and resources for the stream's backing indices.

IMPORTANT: After reloading the search analyzers you should clear the request cache to make sure it doesn't contain responses derived from the previous versions of the analyzer.

You can use the reload search analyzers API to pick up changes to synonym files used in the synonym_graph or synonym token filter of a search analyzer. To be eligible, the token filter must have an updateable flag of true and only be used in search analyzers.

NOTE: This API does not perform a reload for each shard of an index. Instead, it performs a reload for each node containing index shards. As a result, the total shard count returned by the API can differ from the number of index shards. Because reloading affects every node with an index shard, it is important to update the synonym file on every data node in the cluster--including nodes that don't contain a shard replica--before using this API. This ensures the synonym file is updated everywhere in the cluster in case shards are relocated in the future.

Required authorization

  • Index privileges: manage
External documentation

Path parameters

  • index string | array[string] Required

    A comma-separated list of index names to reload analyzers for

Query parameters

  • allow_no_indices boolean

    Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)

  • expand_wildcards string | array[string]

    Whether to expand wildcard expression to concrete indices that are open, closed or both.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    Whether specified concrete indices should be ignored when unavailable (missing or closed)

  • resource string

    Changed resource to reload analyzers from if applicable

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • reload_details array[object] Required
      Hide reload_details attributes Show reload_details attributes object
      • index string Required
      • reloaded_analyzers array[string] Required
      • reloaded_node_ids array[string] Required
    • _shards object Required
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number Required
        • status string
      • skipped number
POST /{index}/_reload_search_analyzers
curl \
 --request POST 'http://api.example.com/{index}/_reload_search_analyzers' \
 --header "Authorization: $API_KEY"

Resolve the cluster Generally available; Added in 8.13.0

GET /_resolve/cluster

Resolve the specified index expressions to return information about each cluster, including the local "querying" cluster, if included. If no index expression is provided, the API will return information about all the remote clusters that are configured on the querying cluster.

This endpoint is useful before doing a cross-cluster search in order to determine which remote clusters should be included in a search.

You use the same index expression with this endpoint as you would for cross-cluster search. Index and cluster exclusions are also supported with this endpoint.

For each cluster in the index expression, information is returned about:

  • Whether the querying ("local") cluster is currently connected to each remote cluster specified in the index expression. Note that this endpoint actively attempts to contact the remote clusters, unlike the remote/info endpoint.
  • Whether each remote cluster is configured with skip_unavailable as true or false.
  • Whether there are any indices, aliases, or data streams on that cluster that match the index expression.
  • Whether the search is likely to have errors returned when you do the cross-cluster search (including any authorization errors if you do not have permission to query the index).
  • Cluster version information, including the Elasticsearch server version.

For example, GET /_resolve/cluster/my-index-*,cluster*:my-index-* returns information about the local cluster and all remotely configured clusters that start with the alias cluster*. Each cluster returns information about whether it has any indices, aliases or data streams that match my-index-*.

Note on backwards compatibility

The ability to query without an index expression was added in version 8.18, so when querying remote clusters older than that, the local cluster will send the index expression dummy* to those remote clusters. Thus, if an errors occur, you may see a reference to that index expression even though you didn't request it. If it causes a problem, you can instead include an index expression like *:* to bypass the issue.

You may want to exclude a cluster or index from a search when:

  • A remote cluster is not currently connected and is configured with skip_unavailable=false. Running a cross-cluster search under those conditions will cause the entire search to fail.
  • A cluster has no matching indices, aliases or data streams for the index expression (or your user does not have permissions to search them). For example, suppose your index expression is logs*,remote1:logs* and the remote1 cluster has no indices, aliases or data streams that match logs*. In that case, that cluster will return no results from that cluster if you include it in a cross-cluster search.
  • The index expression (combined with any query parameters you specify) will likely cause an exception to be thrown when you do the search. In these cases, the "error" field in the _resolve/cluster response will be present. (This is also where security/permission errors will be shown.)
  • A remote cluster is an older version that does not support the feature you want to use in your search.

Test availability of remote clusters

The remote/info endpoint is commonly used to test whether the "local" cluster (the cluster being queried) is connected to its remote clusters, but it does not necessarily reflect whether the remote cluster is available or not. The remote cluster may be available, while the local cluster is not currently connected to it.

You can use the _resolve/cluster API to attempt to reconnect to remote clusters. For example with GET _resolve/cluster or GET _resolve/cluster/*:*. The connected field in the response will indicate whether it was successful. If a connection was (re-)established, this will also cause the remote/info endpoint to now indicate a connected status.

Required authorization

  • Index privileges: view_index_metadata

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar. NOTE: This option is only supported when specifying an index expression. You will get an error if you specify index options to the _resolve/cluster API endpoint that takes no index expression.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. NOTE: This option is only supported when specifying an index expression. You will get an error if you specify index options to the _resolve/cluster API endpoint that takes no index expression.

    Values are all, open, closed, hidden, or none.

  • ignore_throttled boolean Deprecated

    If true, concrete, expanded, or aliased indices are ignored when frozen. NOTE: This option is only supported when specifying an index expression. You will get an error if you specify index options to the _resolve/cluster API endpoint that takes no index expression.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index. NOTE: This option is only supported when specifying an index expression. You will get an error if you specify index options to the _resolve/cluster API endpoint that takes no index expression.

  • timeout string

    The maximum time to wait for remote clusters to respond. If a remote cluster does not respond within this timeout period, the API response will show the cluster as not connected and include an error message that the request timed out.

    The default timeout is unset and the query can take as long as the networking layer is configured to wait for remote clusters that are not responding (typically 30 seconds).

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties

      Provides information about each cluster request relevant to doing a cross-cluster search.

      Hide * attributes Show * attributes object
      • connected boolean Required

        Whether the remote cluster is connected to the local (querying) cluster.

      • skip_unavailable boolean Required

        The skip_unavailable setting for a remote cluster.

      • matching_indices boolean

        Whether the index expression provided in the request matches any indices, aliases or data streams on the cluster.

      • error string

        Provides error messages that are likely to occur if you do a search with this index expression on the specified cluster (for example, lack of security privileges to query an index).

      • version object

        Reduced (minimal) info ElasticsearchVersion

        Hide version attributes Show version attributes object
        • build_flavor string Required
        • minimum_index_compatibility_version string Required
        • minimum_wire_compatibility_version string Required
        • number string Required
GET /_resolve/cluster
curl \
 --request GET 'http://api.example.com/_resolve/cluster' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_resolve/cluster/my-index*,clust*:my-index*`. Each cluster has its own response section. The cluster you sent the request to is labelled as "(local)".
{
  "(local)": {
    "connected": true,
    "skip_unavailable": false,
    "matching_indices": true,
    "version": {
      "number": "8.13.0",
      "build_flavor": "default",
      "minimum_wire_compatibility_version": "7.17.0",
      "minimum_index_compatibility_version": "7.0.0"
    }
  },
  "cluster_one": {
    "connected": true,
    "skip_unavailable": true,
    "matching_indices": true,
    "version": {
      "number": "8.13.0",
      "build_flavor": "default",
      "minimum_wire_compatibility_version": "7.17.0",
      "minimum_index_compatibility_version": "7.0.0"
    }
  },
  "cluster_two": {
    "connected": true,
    "skip_unavailable": false,
    "matching_indices": true,
    "version": {
      "number": "8.13.0",
      "build_flavor": "default",
      "minimum_wire_compatibility_version": "7.17.0",
      "minimum_index_compatibility_version": "7.0.0"
    }
  }
}
A successful response from `GET /_resolve/cluster/not-present,clust*:my-index*,oldcluster:*?ignore_unavailable=false&timeout=5s`. This type of request can be used to identify potential problems with your cross-cluster search. Note also that a `timeout` of 5 seconds is sent, which sets the maximum time the query will wait for remote clusters to respond. The local cluster has no index called `not_present`. Searching with `ignore_unavailable=false` would return a "no such index" error. The `cluster_one` remote cluster has no indices that match the pattern `my-index*`. There may be no indices that match the pattern or the index could be closed. The `cluster_two` remote cluster is not connected (the attempt to connect failed). Since this cluster is marked as `skip_unavailable=false`, you should probably exclude this cluster from the search by adding `-cluster_two:*` to the search index expression. For `cluster_three`, the error message indicates that this remote cluster did not respond within the 5-second timeout window specified, so it is also marked as not connected. The `oldcluster` remote cluster shows that it has matching indices, but no version information is included. This indicates that the cluster version predates the introduction of the `_resolve/cluster` API, so you may want to exclude it from your cross-cluster search.
{
  "(local)": {
    "connected": true,
    "skip_unavailable": false,
    "error": "no such index [not_present]"
  },
  "cluster_one": {
    "connected": true,
    "skip_unavailable": true,
    "matching_indices": false,
    "version": {
      "number": "8.13.0",
      "build_flavor": "default",
      "minimum_wire_compatibility_version": "7.17.0",
      "minimum_index_compatibility_version": "7.0.0"
    }
  },
  "cluster_two": {
    "connected": false,
    "skip_unavailable": false
  },
  "cluster_three": {
    "connected": false,
    "skip_unavailable": false,
    "error": "Request timed out before receiving a response from the remote cluster"
  },
  "oldcluster": {
    "connected": true,
    "skip_unavailable": false,
    "matching_indices": true
  }
}

Resolve the cluster Generally available; Added in 8.13.0

GET /_resolve/cluster/{name}

Resolve the specified index expressions to return information about each cluster, including the local "querying" cluster, if included. If no index expression is provided, the API will return information about all the remote clusters that are configured on the querying cluster.

This endpoint is useful before doing a cross-cluster search in order to determine which remote clusters should be included in a search.

You use the same index expression with this endpoint as you would for cross-cluster search. Index and cluster exclusions are also supported with this endpoint.

For each cluster in the index expression, information is returned about:

  • Whether the querying ("local") cluster is currently connected to each remote cluster specified in the index expression. Note that this endpoint actively attempts to contact the remote clusters, unlike the remote/info endpoint.
  • Whether each remote cluster is configured with skip_unavailable as true or false.
  • Whether there are any indices, aliases, or data streams on that cluster that match the index expression.
  • Whether the search is likely to have errors returned when you do the cross-cluster search (including any authorization errors if you do not have permission to query the index).
  • Cluster version information, including the Elasticsearch server version.

For example, GET /_resolve/cluster/my-index-*,cluster*:my-index-* returns information about the local cluster and all remotely configured clusters that start with the alias cluster*. Each cluster returns information about whether it has any indices, aliases or data streams that match my-index-*.

Note on backwards compatibility

The ability to query without an index expression was added in version 8.18, so when querying remote clusters older than that, the local cluster will send the index expression dummy* to those remote clusters. Thus, if an errors occur, you may see a reference to that index expression even though you didn't request it. If it causes a problem, you can instead include an index expression like *:* to bypass the issue.

You may want to exclude a cluster or index from a search when:

  • A remote cluster is not currently connected and is configured with skip_unavailable=false. Running a cross-cluster search under those conditions will cause the entire search to fail.
  • A cluster has no matching indices, aliases or data streams for the index expression (or your user does not have permissions to search them). For example, suppose your index expression is logs*,remote1:logs* and the remote1 cluster has no indices, aliases or data streams that match logs*. In that case, that cluster will return no results from that cluster if you include it in a cross-cluster search.
  • The index expression (combined with any query parameters you specify) will likely cause an exception to be thrown when you do the search. In these cases, the "error" field in the _resolve/cluster response will be present. (This is also where security/permission errors will be shown.)
  • A remote cluster is an older version that does not support the feature you want to use in your search.

Test availability of remote clusters

The remote/info endpoint is commonly used to test whether the "local" cluster (the cluster being queried) is connected to its remote clusters, but it does not necessarily reflect whether the remote cluster is available or not. The remote cluster may be available, while the local cluster is not currently connected to it.

You can use the _resolve/cluster API to attempt to reconnect to remote clusters. For example with GET _resolve/cluster or GET _resolve/cluster/*:*. The connected field in the response will indicate whether it was successful. If a connection was (re-)established, this will also cause the remote/info endpoint to now indicate a connected status.

Required authorization

  • Index privileges: view_index_metadata

Path parameters

  • name string | array[string] Required

    A comma-separated list of names or index patterns for the indices, aliases, and data streams to resolve. Resources on remote clusters can be specified using the <cluster>:<name> syntax. Index and cluster exclusions (e.g., -cluster1:*) are also supported. If no index expression is specified, information about all remote clusters configured on the local cluster is returned without doing any index matching

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar. NOTE: This option is only supported when specifying an index expression. You will get an error if you specify index options to the _resolve/cluster API endpoint that takes no index expression.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. NOTE: This option is only supported when specifying an index expression. You will get an error if you specify index options to the _resolve/cluster API endpoint that takes no index expression.

    Values are all, open, closed, hidden, or none.

  • ignore_throttled boolean Deprecated

    If true, concrete, expanded, or aliased indices are ignored when frozen. NOTE: This option is only supported when specifying an index expression. You will get an error if you specify index options to the _resolve/cluster API endpoint that takes no index expression.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index. NOTE: This option is only supported when specifying an index expression. You will get an error if you specify index options to the _resolve/cluster API endpoint that takes no index expression.

  • timeout string

    The maximum time to wait for remote clusters to respond. If a remote cluster does not respond within this timeout period, the API response will show the cluster as not connected and include an error message that the request timed out.

    The default timeout is unset and the query can take as long as the networking layer is configured to wait for remote clusters that are not responding (typically 30 seconds).

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties

      Provides information about each cluster request relevant to doing a cross-cluster search.

      Hide * attributes Show * attributes object
      • connected boolean Required

        Whether the remote cluster is connected to the local (querying) cluster.

      • skip_unavailable boolean Required

        The skip_unavailable setting for a remote cluster.

      • matching_indices boolean

        Whether the index expression provided in the request matches any indices, aliases or data streams on the cluster.

      • error string

        Provides error messages that are likely to occur if you do a search with this index expression on the specified cluster (for example, lack of security privileges to query an index).

      • version object

        Reduced (minimal) info ElasticsearchVersion

        Hide version attributes Show version attributes object
        • build_flavor string Required
        • minimum_index_compatibility_version string Required
        • minimum_wire_compatibility_version string Required
        • number string Required
GET /_resolve/cluster/{name}
curl \
 --request GET 'http://api.example.com/_resolve/cluster/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_resolve/cluster/my-index*,clust*:my-index*`. Each cluster has its own response section. The cluster you sent the request to is labelled as "(local)".
{
  "(local)": {
    "connected": true,
    "skip_unavailable": false,
    "matching_indices": true,
    "version": {
      "number": "8.13.0",
      "build_flavor": "default",
      "minimum_wire_compatibility_version": "7.17.0",
      "minimum_index_compatibility_version": "7.0.0"
    }
  },
  "cluster_one": {
    "connected": true,
    "skip_unavailable": true,
    "matching_indices": true,
    "version": {
      "number": "8.13.0",
      "build_flavor": "default",
      "minimum_wire_compatibility_version": "7.17.0",
      "minimum_index_compatibility_version": "7.0.0"
    }
  },
  "cluster_two": {
    "connected": true,
    "skip_unavailable": false,
    "matching_indices": true,
    "version": {
      "number": "8.13.0",
      "build_flavor": "default",
      "minimum_wire_compatibility_version": "7.17.0",
      "minimum_index_compatibility_version": "7.0.0"
    }
  }
}
A successful response from `GET /_resolve/cluster/not-present,clust*:my-index*,oldcluster:*?ignore_unavailable=false&timeout=5s`. This type of request can be used to identify potential problems with your cross-cluster search. Note also that a `timeout` of 5 seconds is sent, which sets the maximum time the query will wait for remote clusters to respond. The local cluster has no index called `not_present`. Searching with `ignore_unavailable=false` would return a "no such index" error. The `cluster_one` remote cluster has no indices that match the pattern `my-index*`. There may be no indices that match the pattern or the index could be closed. The `cluster_two` remote cluster is not connected (the attempt to connect failed). Since this cluster is marked as `skip_unavailable=false`, you should probably exclude this cluster from the search by adding `-cluster_two:*` to the search index expression. For `cluster_three`, the error message indicates that this remote cluster did not respond within the 5-second timeout window specified, so it is also marked as not connected. The `oldcluster` remote cluster shows that it has matching indices, but no version information is included. This indicates that the cluster version predates the introduction of the `_resolve/cluster` API, so you may want to exclude it from your cross-cluster search.
{
  "(local)": {
    "connected": true,
    "skip_unavailable": false,
    "error": "no such index [not_present]"
  },
  "cluster_one": {
    "connected": true,
    "skip_unavailable": true,
    "matching_indices": false,
    "version": {
      "number": "8.13.0",
      "build_flavor": "default",
      "minimum_wire_compatibility_version": "7.17.0",
      "minimum_index_compatibility_version": "7.0.0"
    }
  },
  "cluster_two": {
    "connected": false,
    "skip_unavailable": false
  },
  "cluster_three": {
    "connected": false,
    "skip_unavailable": false,
    "error": "Request timed out before receiving a response from the remote cluster"
  },
  "oldcluster": {
    "connected": true,
    "skip_unavailable": false,
    "matching_indices": true
  }
}




Roll over to a new index Generally available; Added in 5.0.0

POST /{alias}/_rollover

TIP: It is recommended to use the index lifecycle rollover action to automate rollovers.

The rollover API creates a new index for a data stream or index alias. The API behavior depends on the rollover target.

Roll over a data stream

If you roll over a data stream, the API creates a new write index for the stream. The stream's previous write index becomes a regular backing index. A rollover also increments the data stream's generation.

Roll over an index alias with a write index

TIP: Prior to Elasticsearch 7.9, you'd typically use an index alias with a write index to manage time series data. Data streams replace this functionality, require less maintenance, and automatically integrate with data tiers.

If an index alias points to multiple indices, one of the indices must be a write index. The rollover API creates a new write index for the alias with is_write_index set to true. The API also sets is_write_index to false for the previous write index.

Roll over an index alias with one index

If you roll over an index alias that points to only one index, the API creates a new index for the alias and removes the original index from the alias.

NOTE: A rollover creates a new index and is subject to the wait_for_active_shards setting.

Increment index names for an alias

When you roll over an index alias, you can specify a name for the new index. If you don't specify a name and the current index ends with - and a number, such as my-index-000001 or my-index-3, the new index name increments that number. For example, if you roll over an alias with a current index of my-index-000001, the rollover creates a new index named my-index-000002. This number is always six characters and zero-padded, regardless of the previous index's name.

If you use an index alias for time series data, you can use date math in the index name to track the rollover date. For example, you can create an alias that points to an index named <my-index-{now/d}-000001>. If you create the index on May 6, 2099, the index's name is my-index-2099.05.06-000001. If you roll over the alias on May 7, 2099, the new index's name is my-index-2099.05.07-000002.

Required authorization

  • Index privileges: manage

Path parameters

  • alias string Required

    Name of the data stream or index alias to roll over.

Query parameters

  • dry_run boolean

    If true, checks whether the current index satisfies the specified conditions but does not perform a rollover.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).

    Values are all or index-setting.

  • lazy boolean

    If set to true, the rollover action will only mark a data stream to signal that it needs to be rolled over at the next write. Only allowed on data streams.

application/json

Body

  • aliases object

    Aliases for the target index. Data streams do not support this parameter.

    Hide aliases attribute Show aliases attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • filter object

        An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

        External documentation
      • index_routing string
      • is_hidden boolean

        If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

      • is_write_index boolean

        If true, the index is the write index for the alias.

      • routing string
      • search_routing string
  • conditions object
    Hide conditions attributes Show conditions attributes object
    • min_age string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • max_age string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • max_age_millis number

      Time unit for milliseconds

    • min_docs number
    • max_docs number
    • max_size number | string

    • max_size_bytes number
    • min_size number | string

    • min_size_bytes number
    • max_primary_shard_size number | string

    • max_primary_shard_size_bytes number
    • min_primary_shard_size number | string

    • min_primary_shard_size_bytes number
    • max_primary_shard_docs number
    • min_primary_shard_docs number
  • mappings object
    Hide mappings attributes Show mappings attributes object
    • all_field object
      Hide all_field attributes Show all_field attributes object
      • analyzer string Required
      • enabled boolean Required
      • omit_norms boolean Required
      • search_analyzer string Required
      • similarity string Required
      • store boolean Required
      • store_term_vector_offsets boolean Required
      • store_term_vector_payloads boolean Required
      • store_term_vector_positions boolean Required
      • store_term_vectors boolean Required
    • date_detection boolean
    • dynamic string

      Values are strict, runtime, true, or false.

    • dynamic_date_formats array[string]
    • dynamic_templates array[object]
    • _field_names object
      Hide _field_names attribute Show _field_names attribute object
      • enabled boolean Required
    • index_field object
      Hide index_field attribute Show index_field attribute object
      • enabled boolean Required
    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties
    • numeric_detection boolean
    • properties object
    • _routing object
      Hide _routing attribute Show _routing attribute object
      • required boolean Required
    • _size object
      Hide _size attribute Show _size attribute object
      • enabled boolean Required
    • _source object
      Hide _source attributes Show _source attributes object
      • compress boolean
      • compress_threshold string
      • enabled boolean
      • excludes array[string]
      • includes array[string]
      • mode string

        Values are disabled, stored, or synthetic.

    • runtime object
      Hide runtime attribute Show runtime attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • format string
        • format string

          A custom format for date type runtime fields.

        • input_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_index string
        • script object
          Hide script attributes Show script attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties
          • lang string

            Any of:

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • type string Required

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • enabled boolean
    • subobjects string

      Values are true or false.

    • _data_stream_timestamp object
      Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
      • enabled boolean Required
  • settings object

    Configuration options for the index. Data streams do not support this parameter.

    Hide settings attribute Show settings attribute object
    • * object Additional properties

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required
    • conditions object Required
      Hide conditions attribute Show conditions attribute object
      • * boolean Additional properties
    • dry_run boolean Required
    • new_index string Required
    • old_index string Required
    • rolled_over boolean Required
    • shards_acknowledged boolean Required
POST /{alias}/_rollover
curl \
 --request POST 'http://api.example.com/{alias}/_rollover' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"conditions\": {\n    \"max_age\": \"7d\",\n    \"max_docs\": 1000,\n    \"max_primary_shard_size\": \"50gb\",\n    \"max_primary_shard_docs\": \"2000\"\n  }\n}"'
Request example
{
  "conditions": {
    "max_age": "7d",
    "max_docs": 1000,
    "max_primary_shard_size": "50gb",
    "max_primary_shard_docs": "2000"
  }
}
Response examples (200)
An abbreviated response from `GET /_segments`.
{
  "_shards": {},
  "indices": {
    "test": {
      "shards": {
        "0": [
          {
            "routing": {
              "state": "STARTED",
              "primary": true,
              "node": "zDC_RorJQCao9xf9pg3Fvw"
            },
            "num_committed_segments": 0,
            "num_search_segments": 1,
            "segments": {
              "_0": {
                "generation": 0,
                "num_docs": 1,
                "deleted_docs": 0,
                "size_in_bytes": 3800,
                "committed": false,
                "search": true,
                "version": "7.0.0",
                "compound": true,
                "attributes": {}
              }
            }
          }
        ]
      }
    }
  }
}

Roll over to a new index Generally available; Added in 5.0.0

POST /{alias}/_rollover/{new_index}

TIP: It is recommended to use the index lifecycle rollover action to automate rollovers.

The rollover API creates a new index for a data stream or index alias. The API behavior depends on the rollover target.

Roll over a data stream

If you roll over a data stream, the API creates a new write index for the stream. The stream's previous write index becomes a regular backing index. A rollover also increments the data stream's generation.

Roll over an index alias with a write index

TIP: Prior to Elasticsearch 7.9, you'd typically use an index alias with a write index to manage time series data. Data streams replace this functionality, require less maintenance, and automatically integrate with data tiers.

If an index alias points to multiple indices, one of the indices must be a write index. The rollover API creates a new write index for the alias with is_write_index set to true. The API also sets is_write_index to false for the previous write index.

Roll over an index alias with one index

If you roll over an index alias that points to only one index, the API creates a new index for the alias and removes the original index from the alias.

NOTE: A rollover creates a new index and is subject to the wait_for_active_shards setting.

Increment index names for an alias

When you roll over an index alias, you can specify a name for the new index. If you don't specify a name and the current index ends with - and a number, such as my-index-000001 or my-index-3, the new index name increments that number. For example, if you roll over an alias with a current index of my-index-000001, the rollover creates a new index named my-index-000002. This number is always six characters and zero-padded, regardless of the previous index's name.

If you use an index alias for time series data, you can use date math in the index name to track the rollover date. For example, you can create an alias that points to an index named <my-index-{now/d}-000001>. If you create the index on May 6, 2099, the index's name is my-index-2099.05.06-000001. If you roll over the alias on May 7, 2099, the new index's name is my-index-2099.05.07-000002.

Required authorization

  • Index privileges: manage

Path parameters

  • alias string Required

    Name of the data stream or index alias to roll over.

  • new_index string Required

    Name of the index to create. Supports date math. Data streams do not support this parameter.

Query parameters

  • dry_run boolean

    If true, checks whether the current index satisfies the specified conditions but does not perform a rollover.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).

    Values are all or index-setting.

  • lazy boolean

    If set to true, the rollover action will only mark a data stream to signal that it needs to be rolled over at the next write. Only allowed on data streams.

application/json

Body

  • aliases object

    Aliases for the target index. Data streams do not support this parameter.

    Hide aliases attribute Show aliases attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • filter object

        An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

        External documentation
      • index_routing string
      • is_hidden boolean

        If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

      • is_write_index boolean

        If true, the index is the write index for the alias.

      • routing string
      • search_routing string
  • conditions object
    Hide conditions attributes Show conditions attributes object
    • min_age string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • max_age string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • max_age_millis number

      Time unit for milliseconds

    • min_docs number
    • max_docs number
    • max_size number | string

    • max_size_bytes number
    • min_size number | string

    • min_size_bytes number
    • max_primary_shard_size number | string

    • max_primary_shard_size_bytes number
    • min_primary_shard_size number | string

    • min_primary_shard_size_bytes number
    • max_primary_shard_docs number
    • min_primary_shard_docs number
  • mappings object
    Hide mappings attributes Show mappings attributes object
    • all_field object
      Hide all_field attributes Show all_field attributes object
      • analyzer string Required
      • enabled boolean Required
      • omit_norms boolean Required
      • search_analyzer string Required
      • similarity string Required
      • store boolean Required
      • store_term_vector_offsets boolean Required
      • store_term_vector_payloads boolean Required
      • store_term_vector_positions boolean Required
      • store_term_vectors boolean Required
    • date_detection boolean
    • dynamic string

      Values are strict, runtime, true, or false.

    • dynamic_date_formats array[string]
    • dynamic_templates array[object]
    • _field_names object
      Hide _field_names attribute Show _field_names attribute object
      • enabled boolean Required
    • index_field object
      Hide index_field attribute Show index_field attribute object
      • enabled boolean Required
    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties
    • numeric_detection boolean
    • properties object
    • _routing object
      Hide _routing attribute Show _routing attribute object
      • required boolean Required
    • _size object
      Hide _size attribute Show _size attribute object
      • enabled boolean Required
    • _source object
      Hide _source attributes Show _source attributes object
      • compress boolean
      • compress_threshold string
      • enabled boolean
      • excludes array[string]
      • includes array[string]
      • mode string

        Values are disabled, stored, or synthetic.

    • runtime object
      Hide runtime attribute Show runtime attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • format string
        • format string

          A custom format for date type runtime fields.

        • input_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_index string
        • script object
          Hide script attributes Show script attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties
          • lang string

            Any of:

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • type string Required

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • enabled boolean
    • subobjects string

      Values are true or false.

    • _data_stream_timestamp object
      Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
      • enabled boolean Required
  • settings object

    Configuration options for the index. Data streams do not support this parameter.

    Hide settings attribute Show settings attribute object
    • * object Additional properties

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required
    • conditions object Required
      Hide conditions attribute Show conditions attribute object
      • * boolean Additional properties
    • dry_run boolean Required
    • new_index string Required
    • old_index string Required
    • rolled_over boolean Required
    • shards_acknowledged boolean Required
POST /{alias}/_rollover/{new_index}
curl \
 --request POST 'http://api.example.com/{alias}/_rollover/{new_index}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"conditions\": {\n    \"max_age\": \"7d\",\n    \"max_docs\": 1000,\n    \"max_primary_shard_size\": \"50gb\",\n    \"max_primary_shard_docs\": \"2000\"\n  }\n}"'
Request example
{
  "conditions": {
    "max_age": "7d",
    "max_docs": 1000,
    "max_primary_shard_size": "50gb",
    "max_primary_shard_docs": "2000"
  }
}
Response examples (200)
An abbreviated response from `GET /_segments`.
{
  "_shards": {},
  "indices": {
    "test": {
      "shards": {
        "0": [
          {
            "routing": {
              "state": "STARTED",
              "primary": true,
              "node": "zDC_RorJQCao9xf9pg3Fvw"
            },
            "num_committed_segments": 0,
            "num_search_segments": 1,
            "segments": {
              "_0": {
                "generation": 0,
                "num_docs": 1,
                "deleted_docs": 0,
                "size_in_bytes": 3800,
                "committed": false,
                "search": true,
                "version": "7.0.0",
                "compound": true,
                "attributes": {}
              }
            }
          }
        ]
      }
    }
  }
}

Get index segments Generally available

GET /_segments

Get low-level information about the Lucene segments in index shards. For data streams, the API returns information about the stream's backing indices.

Required authorization

  • Index privileges: monitor

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • verbose boolean

    If true, the request returns a verbose response.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • indices object Required
      Hide indices attribute Show indices attribute object
      • * object Additional properties
        Hide * attribute Show * attribute object
        • shards object Required
    • _shards object Required
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number Required
        • status string
      • skipped number
GET /_segments
curl \
 --request GET 'http://api.example.com/_segments' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response for creating a new index for a data stream.
{
  "acknowledged": true,
  "shards_acknowledged": true,
  "old_index": ".ds-my-data-stream-2099.05.06-000001",
  "new_index": ".ds-my-data-stream-2099.05.07-000002",
  "rolled_over": true,
  "dry_run": false,
  "lazy": false,
  "conditions": {
    "[max_age: 7d]": false,
    "[max_docs: 1000]": true,
    "[max_primary_shard_size: 50gb]": false,
    "[max_primary_shard_docs: 2000]": false
  }
}




Get index shard stores Generally available

GET /_shard_stores

Get store information about replica shards in one or more indices. For data streams, the API retrieves store information for the stream's backing indices.

The index shard stores API returns the following information:

  • The node on which each replica shard exists.
  • The allocation ID for each replica shard.
  • A unique ID for each replica shard.
  • Any errors encountered while opening the shard index or from an earlier failure.

By default, the API returns store information only for primary shards that are unassigned or have one or more unassigned replica shards.

Required authorization

  • Index privileges: monitor

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If true, missing or closed indices are not included in the response.

  • status string | array[string]

    List of shard health statuses used to limit the request.

    Values are green, yellow, red, or all.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • indices object Required
      Hide indices attribute Show indices attribute object
      • * object Additional properties
        Hide * attribute Show * attribute object
        • shards object Required
          Hide shards attribute Show shards attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • stores array[object] Required
GET /_shard_stores
curl \
 --request GET 'http://api.example.com/_shard_stores' \
 --header "Authorization: $API_KEY"
Response examples (200)
An abbreviated response from `GET /_shard_stores?status=green`.
{
  "indices": {
    "my-index-000001": {
      "shards": {
        "0": {
          "stores": [
            {
              "sPa3OgxLSYGvQ4oPs-Tajw": {
                "name": "node_t0",
                "ephemeral_id": "9NlXRFGCT1m8tkvYCMK-8A",
                "transport_address": "local[1]",
                "external_id": "node_t0",
                "attributes": {},
                "roles": [],
                "version": "8.10.0",
                "min_index_version": 7000099,
                "max_index_version": 8100099
              },
              "allocation_id": "2iNySv_OQVePRX-yaRH_lQ",
              "allocation": "primary",
              "store_exception": {}
            }
          ]
        }
      }
    }
  }
}

Get index shard stores Generally available

GET /{index}/_shard_stores

Get store information about replica shards in one or more indices. For data streams, the API retrieves store information for the stream's backing indices.

The index shard stores API returns the following information:

  • The node on which each replica shard exists.
  • The allocation ID for each replica shard.
  • A unique ID for each replica shard.
  • Any errors encountered while opening the shard index or from an earlier failure.

By default, the API returns store information only for primary shards that are unassigned or have one or more unassigned replica shards.

Required authorization

  • Index privileges: monitor

Path parameters

  • index string | array[string] Required

    List of data streams, indices, and aliases used to limit the request.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.

    Values are all, open, closed, hidden, or none.

  • ignore_unavailable boolean

    If true, missing or closed indices are not included in the response.

  • status string | array[string]

    List of shard health statuses used to limit the request.

    Values are green, yellow, red, or all.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • indices object Required
      Hide indices attribute Show indices attribute object
      • * object Additional properties
        Hide * attribute Show * attribute object
        • shards object Required
          Hide shards attribute Show shards attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • stores array[object] Required
GET /{index}/_shard_stores
curl \
 --request GET 'http://api.example.com/{index}/_shard_stores' \
 --header "Authorization: $API_KEY"
Response examples (200)
An abbreviated response from `GET /_shard_stores?status=green`.
{
  "indices": {
    "my-index-000001": {
      "shards": {
        "0": {
          "stores": [
            {
              "sPa3OgxLSYGvQ4oPs-Tajw": {
                "name": "node_t0",
                "ephemeral_id": "9NlXRFGCT1m8tkvYCMK-8A",
                "transport_address": "local[1]",
                "external_id": "node_t0",
                "attributes": {},
                "roles": [],
                "version": "8.10.0",
                "min_index_version": 7000099,
                "max_index_version": 8100099
              },
              "allocation_id": "2iNySv_OQVePRX-yaRH_lQ",
              "allocation": "primary",
              "store_exception": {}
            }
          ]
        }
      }
    }
  }
}

Shrink an index Generally available; Added in 5.0.0

PUT /{index}/_shrink/{target}

Shrink an index into a new index with fewer primary shards.

Before you can shrink an index:

  • The index must be read-only.
  • A copy of every shard in the index must reside on the same node.
  • The index must have a green health status.

To make shard allocation easier, we recommend you also remove the index's replica shards. You can later re-add replica shards as part of the shrink operation.

The requested number of primary shards in the target index must be a factor of the number of shards in the source index. For example an index with 8 primary shards can be shrunk into 4, 2 or 1 primary shards or an index with 15 primary shards can be shrunk into 5, 3 or 1. If the number of shards in the index is a prime number it can only be shrunk into a single primary shard Before shrinking, a (primary or replica) copy of every shard in the index must be present on the same node.

The current write index on a data stream cannot be shrunk. In order to shrink the current write index, the data stream must first be rolled over so that a new write index is created and then the previous write index can be shrunk.

A shrink operation:

  • Creates a new target index with the same definition as the source index, but with a smaller number of primary shards.
  • Hard-links segments from the source index into the target index. If the file system does not support hard-linking, then all segments are copied into the new index, which is a much more time consuming process. Also if using multiple data paths, shards on different data paths require a full copy of segment files if they are not on the same disk since hardlinks do not work across disks.
  • Recovers the target index as though it were a closed index which had just been re-opened. Recovers shards to the .routing.allocation.initial_recovery._id index setting.

IMPORTANT: Indices can only be shrunk if they satisfy the following requirements:

  • The target index must not exist.
  • The source index must have more primary shards than the target index.
  • The number of primary shards in the target index must be a factor of the number of primary shards in the source index. The source index must have more primary shards than the target index.
  • The index must not contain more than 2,147,483,519 documents in total across all shards that will be shrunk into a single shard on the target index as this is the maximum number of docs that can fit into a single shard.
  • The node handling the shrink process must have sufficient free disk space to accommodate a second copy of the existing index.

Required authorization

  • Index privileges: manage

Path parameters

  • index string Required

    Name of the source index to shrink.

  • target string Required

    Name of the target index to create.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).

    Values are all or index-setting.

application/json

Body

  • aliases object

    The key is the alias name. Index alias names support date math.

    Hide aliases attribute Show aliases attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • filter object

        An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

        External documentation
      • index_routing string
      • is_hidden boolean

        If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

      • is_write_index boolean

        If true, the index is the write index for the alias.

      • routing string
      • search_routing string
  • settings object

    Configuration options for the target index.

    Hide settings attribute Show settings attribute object
    • * object Additional properties

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required
    • shards_acknowledged boolean Required
    • index string Required
PUT /{index}/_shrink/{target}
curl \
 --request PUT 'http://api.example.com/{index}/_shrink/{target}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"settings\": {\n    \"index.routing.allocation.require._name\": null,\n    \"index.blocks.write\": null\n  }\n}"'
Request example
{
  "settings": {
    "index.routing.allocation.require._name": null,
    "index.blocks.write": null
  }
}

Shrink an index Generally available; Added in 5.0.0

POST /{index}/_shrink/{target}

Shrink an index into a new index with fewer primary shards.

Before you can shrink an index:

  • The index must be read-only.
  • A copy of every shard in the index must reside on the same node.
  • The index must have a green health status.

To make shard allocation easier, we recommend you also remove the index's replica shards. You can later re-add replica shards as part of the shrink operation.

The requested number of primary shards in the target index must be a factor of the number of shards in the source index. For example an index with 8 primary shards can be shrunk into 4, 2 or 1 primary shards or an index with 15 primary shards can be shrunk into 5, 3 or 1. If the number of shards in the index is a prime number it can only be shrunk into a single primary shard Before shrinking, a (primary or replica) copy of every shard in the index must be present on the same node.

The current write index on a data stream cannot be shrunk. In order to shrink the current write index, the data stream must first be rolled over so that a new write index is created and then the previous write index can be shrunk.

A shrink operation:

  • Creates a new target index with the same definition as the source index, but with a smaller number of primary shards.
  • Hard-links segments from the source index into the target index. If the file system does not support hard-linking, then all segments are copied into the new index, which is a much more time consuming process. Also if using multiple data paths, shards on different data paths require a full copy of segment files if they are not on the same disk since hardlinks do not work across disks.
  • Recovers the target index as though it were a closed index which had just been re-opened. Recovers shards to the .routing.allocation.initial_recovery._id index setting.

IMPORTANT: Indices can only be shrunk if they satisfy the following requirements:

  • The target index must not exist.
  • The source index must have more primary shards than the target index.
  • The number of primary shards in the target index must be a factor of the number of primary shards in the source index. The source index must have more primary shards than the target index.
  • The index must not contain more than 2,147,483,519 documents in total across all shards that will be shrunk into a single shard on the target index as this is the maximum number of docs that can fit into a single shard.
  • The node handling the shrink process must have sufficient free disk space to accommodate a second copy of the existing index.

Required authorization

  • Index privileges: manage

Path parameters

  • index string Required

    Name of the source index to shrink.

  • target string Required

    Name of the target index to create.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).

    Values are all or index-setting.

application/json

Body

  • aliases object

    The key is the alias name. Index alias names support date math.

    Hide aliases attribute Show aliases attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • filter object

        An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

        External documentation
      • index_routing string
      • is_hidden boolean

        If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

      • is_write_index boolean

        If true, the index is the write index for the alias.

      • routing string
      • search_routing string
  • settings object

    Configuration options for the target index.

    Hide settings attribute Show settings attribute object
    • * object Additional properties

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required
    • shards_acknowledged boolean Required
    • index string Required
POST /{index}/_shrink/{target}
curl \
 --request POST 'http://api.example.com/{index}/_shrink/{target}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"settings\": {\n    \"index.routing.allocation.require._name\": null,\n    \"index.blocks.write\": null\n  }\n}"'
Request example
{
  "settings": {
    "index.routing.allocation.require._name": null,
    "index.blocks.write": null
  }
}
















Split an index Generally available; Added in 6.1.0

POST /{index}/_split/{target}

Split an index into a new index with more primary shards.

  • Before you can split an index:

  • The index must be read-only.

  • The cluster health status must be green.

You can do make an index read-only with the following request using the add index block API:

PUT /my_source_index/_block/write

The current write index on a data stream cannot be split. In order to split the current write index, the data stream must first be rolled over so that a new write index is created and then the previous write index can be split.

The number of times the index can be split (and the number of shards that each original shard can be split into) is determined by the index.number_of_routing_shards setting. The number of routing shards specifies the hashing space that is used internally to distribute documents across shards with consistent hashing. For instance, a 5 shard index with number_of_routing_shards set to 30 (5 x 2 x 3) could be split by a factor of 2 or 3.

A split operation:

  • Creates a new target index with the same definition as the source index, but with a larger number of primary shards.
  • Hard-links segments from the source index into the target index. If the file system doesn't support hard-linking, all segments are copied into the new index, which is a much more time consuming process.
  • Hashes all documents again, after low level files are created, to delete documents that belong to a different shard.
  • Recovers the target index as though it were a closed index which had just been re-opened.

IMPORTANT: Indices can only be split if they satisfy the following requirements:

  • The target index must not exist.
  • The source index must have fewer primary shards than the target index.
  • The number of primary shards in the target index must be a multiple of the number of primary shards in the source index.
  • The node handling the split process must have sufficient free disk space to accommodate a second copy of the existing index.

Required authorization

  • Index privileges: manage

Path parameters

  • index string Required

    Name of the source index to split.

  • target string Required

    Name of the target index to create.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).

    Values are all or index-setting.

application/json

Body

  • aliases object

    Aliases for the resulting index.

    Hide aliases attribute Show aliases attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • filter object

        An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

        External documentation
      • index_routing string
      • is_hidden boolean

        If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

      • is_write_index boolean

        If true, the index is the write index for the alias.

      • routing string
      • search_routing string
  • settings object

    Configuration options for the target index.

    Hide settings attribute Show settings attribute object
    • * object Additional properties

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required
    • shards_acknowledged boolean Required
    • index string Required
POST /{index}/_split/{target}
curl \
 --request POST 'http://api.example.com/{index}/_split/{target}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"settings\": {\n    \"index.number_of_shards\": 2\n  }\n}"'
Request example
Split an existing index into a new index with more primary shards.
{
  "settings": {
    "index.number_of_shards": 2
  }
}

Get index statistics Generally available; Added in 1.3.0

GET /_stats

For data streams, the API retrieves statistics for the stream's backing indices.

By default, the returned statistics are index-level with primaries and total aggregations. primaries are the values for only the primary shards. total are the accumulated values for both primary and replica shards.

To get shard-level statistics, set the level parameter to shards.

NOTE: When moving to another node, the shard-level statistics for a shard are cleared. Although the shard is no longer part of the node, that node retains any node-level statistics to which the shard contributed.

Required authorization

  • Index privileges: monitor

Query parameters

  • completion_fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in fielddata and suggest statistics.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • fielddata_fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in fielddata statistics.

  • fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in the statistics.

  • forbid_closed_indices boolean

    If true, statistics are not collected from closed indices.

  • groups string | array[string]

    Comma-separated list of search groups to include in the search statistics.

  • include_segment_file_sizes boolean

    If true, the call reports the aggregated disk usage of each one of the Lucene index files (only applies if segment stats are requested).

  • include_unloaded_segments boolean

    If true, the response includes information from segments that are not loaded into memory.

  • level string

    Indicates whether statistics are aggregated at the cluster, index, or shard level.

    Values are cluster, indices, or shards.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • indices object
      Hide indices attribute Show indices attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • primaries object
          Hide primaries attributes Show primaries attributes object
          • completion object
            Hide completion attributes Show completion attributes object
            • size_in_bytes number Required

              Total amount, in bytes, of memory used for completion across all shards assigned to selected nodes.

            • size number | string

            • fields object
              Hide fields attribute Show fields attribute object
              • * object Additional properties
          • docs object
            Hide docs attributes Show docs attributes object
            • count number Required

              Total number of non-deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments and may include documents from nested fields.

            • deleted number

              Total number of deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments. Elasticsearch reclaims the disk space of deleted Lucene documents when a segment is merged.

          • fielddata object
            Hide fielddata attributes Show fielddata attributes object
            • evictions number
            • memory_size number | string

            • memory_size_in_bytes number Required
            • fields object
              Hide fields attribute Show fields attribute object
              • * object Additional properties
          • flush object
            Hide flush attributes Show flush attributes object
            • periodic number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

          • get object
            Hide get attributes Show get attributes object
            • current number Required
            • exists_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • exists_time_in_millis number

              Time unit for milliseconds

            • exists_total number Required
            • missing_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • missing_time_in_millis number

              Time unit for milliseconds

            • missing_total number Required
            • time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • time_in_millis number

              Time unit for milliseconds

            • total number Required
          • indexing object
            Hide indexing attributes Show indexing attributes object
            • index_current number Required
            • delete_current number Required
            • delete_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • delete_time_in_millis number

              Time unit for milliseconds

            • delete_total number Required
            • is_throttled boolean Required
            • noop_update_total number Required
            • throttle_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • throttle_time_in_millis number

              Time unit for milliseconds

            • index_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • index_time_in_millis number

              Time unit for milliseconds

            • index_total number Required
            • index_failed number Required
            • types object
              Hide types attribute Show types attribute object
              • * object
            • write_load number
          • indices object Additional properties
          • merges object
            Hide merges attributes Show merges attributes object
            • current number Required
            • current_docs number Required
            • current_size string
            • current_size_in_bytes number Required
            • total number Required
            • total_auto_throttle string
            • total_auto_throttle_in_bytes number Required
            • total_docs number Required
            • total_size string
            • total_size_in_bytes number Required
            • total_stopped_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_stopped_time_in_millis number

              Time unit for milliseconds

            • total_throttled_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_throttled_time_in_millis number

              Time unit for milliseconds

            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

          • query_cache object
            Hide query_cache attributes Show query_cache attributes object
            • cache_count number Required

              Total number of entries added to the query cache across all shards assigned to selected nodes. This number includes current and evicted entries.

            • cache_size number Required

              Total number of entries currently in the query cache across all shards assigned to selected nodes.

            • evictions number Required

              Total number of query cache evictions across all shards assigned to selected nodes.

            • hit_count number Required

              Total count of query cache hits across all shards assigned to selected nodes.

            • memory_size number | string

            • memory_size_in_bytes number Required

              Total amount, in bytes, of memory used for the query cache across all shards assigned to selected nodes.

            • miss_count number Required

              Total count of query cache misses across all shards assigned to selected nodes.

            • total_count number Required

              Total count of hits and misses in the query cache across all shards assigned to selected nodes.

          • recovery object
            Hide recovery attributes Show recovery attributes object
            • current_as_source number Required
            • current_as_target number Required
            • throttle_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • throttle_time_in_millis number

              Time unit for milliseconds

          • refresh object
            Hide refresh attributes Show refresh attributes object
            • external_total number Required
            • external_total_time_in_millis number

              Time unit for milliseconds

            • listeners number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

          • request_cache object
            Hide request_cache attributes Show request_cache attributes object
            • evictions number Required
            • hit_count number Required
            • memory_size string
            • memory_size_in_bytes number Required
            • miss_count number Required
          • segments object
            Hide segments attributes Show segments attributes object
            • count number Required

              Total number of segments across all shards assigned to selected nodes.

            • doc_values_memory number | string

            • doc_values_memory_in_bytes number Required

              Total amount, in bytes, of memory used for doc values across all shards assigned to selected nodes.

            • file_sizes object Required

              This object is not populated by the cluster stats API. To get information on segment files, use the node stats API.

              Hide file_sizes attribute Show file_sizes attribute object
              • * object Additional properties
            • fixed_bit_set number | string

            • fixed_bit_set_memory_in_bytes number Required

              Total amount of memory, in bytes, used by fixed bit sets across all shards assigned to selected nodes.

            • index_writer_memory number | string

            • index_writer_max_memory_in_bytes number
            • index_writer_memory_in_bytes number Required

              Total amount, in bytes, of memory used by all index writers across all shards assigned to selected nodes.

            • max_unsafe_auto_id_timestamp number Required

              Unix timestamp, in milliseconds, of the most recently retried indexing request.

            • memory number | string

            • memory_in_bytes number Required

              Total amount, in bytes, of memory used for segments across all shards assigned to selected nodes.

            • norms_memory number | string

            • norms_memory_in_bytes number Required

              Total amount, in bytes, of memory used for normalization factors across all shards assigned to selected nodes.

            • points_memory number | string

            • points_memory_in_bytes number Required

              Total amount, in bytes, of memory used for points across all shards assigned to selected nodes.

            • stored_memory number | string

            • stored_fields_memory_in_bytes number Required

              Total amount, in bytes, of memory used for stored fields across all shards assigned to selected nodes.

            • terms_memory_in_bytes number Required

              Total amount, in bytes, of memory used for terms across all shards assigned to selected nodes.

            • terms_memory number | string

            • term_vectory_memory number | string

            • term_vectors_memory_in_bytes number Required

              Total amount, in bytes, of memory used for term vectors across all shards assigned to selected nodes.

            • version_map_memory number | string

            • version_map_memory_in_bytes number Required

              Total amount, in bytes, of memory used by all version maps across all shards assigned to selected nodes.

          • store object
            Hide store attributes Show store attributes object
            • size number | string

            • size_in_bytes number Required

              Total size, in bytes, of all shards assigned to selected nodes.

            • reserved number | string

            • reserved_in_bytes number Required

              A prediction, in bytes, of how much larger the shard stores will eventually grow due to ongoing peer recoveries, restoring snapshots, and similar activities.

            • total_data_set_size number | string

            • total_data_set_size_in_bytes number

              Total data set size, in bytes, of all shards assigned to selected nodes. This includes the size of shards not stored fully on the nodes, such as the cache for partially mounted indices.

          • translog object
            Hide translog attributes Show translog attributes object
            • earliest_last_modified_age number Required
            • operations number Required
            • size string
            • size_in_bytes number Required
            • uncommitted_operations number Required
            • uncommitted_size string
            • uncommitted_size_in_bytes number Required
          • warmer object
            Hide warmer attributes Show warmer attributes object
            • current number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

          • bulk object
            Hide bulk attributes Show bulk attributes object
            • total_operations number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

            • total_size number | string

            • total_size_in_bytes number Required
            • avg_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • avg_time_in_millis number

              Time unit for milliseconds

            • avg_size number | string

            • avg_size_in_bytes number Required
          • shard_stats object
            Hide shard_stats attribute Show shard_stats attribute object
            • total_count number Required
        • shards object
          Hide shards attribute Show shards attribute object
          • * array[object] Additional properties
            Hide * attributes Show * attributes object
            • commit object
            • completion object
            • docs object
            • fielddata object
            • flush object
            • get object
            • indexing object
            • mappings object
            • merges object
            • shard_path object
            • query_cache object
            • recovery object
            • refresh object
            • request_cache object
            • retention_leases object
            • routing object
            • segments object
            • seq_no object
            • store object
            • translog object
            • warmer object
            • bulk object
            • shards object Generally available; Added in 7.15.0
            • shard_stats object
            • indices object Additional properties
        • total object
          Hide total attributes Show total attributes object
          • completion object
            Hide completion attributes Show completion attributes object
            • size_in_bytes number Required

              Total amount, in bytes, of memory used for completion across all shards assigned to selected nodes.

            • size number | string

            • fields object
              Hide fields attribute Show fields attribute object
              • * object Additional properties
          • docs object
            Hide docs attributes Show docs attributes object
            • count number Required

              Total number of non-deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments and may include documents from nested fields.

            • deleted number

              Total number of deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments. Elasticsearch reclaims the disk space of deleted Lucene documents when a segment is merged.

          • fielddata object
            Hide fielddata attributes Show fielddata attributes object
            • evictions number
            • memory_size number | string

            • memory_size_in_bytes number Required
            • fields object
              Hide fields attribute Show fields attribute object
              • * object Additional properties
          • flush object
            Hide flush attributes Show flush attributes object
            • periodic number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

          • get object
            Hide get attributes Show get attributes object
            • current number Required
            • exists_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • exists_time_in_millis number

              Time unit for milliseconds

            • exists_total number Required
            • missing_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • missing_time_in_millis number

              Time unit for milliseconds

            • missing_total number Required
            • time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • time_in_millis number

              Time unit for milliseconds

            • total number Required
          • indexing object
            Hide indexing attributes Show indexing attributes object
            • index_current number Required
            • delete_current number Required
            • delete_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • delete_time_in_millis number

              Time unit for milliseconds

            • delete_total number Required
            • is_throttled boolean Required
            • noop_update_total number Required
            • throttle_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • throttle_time_in_millis number

              Time unit for milliseconds

            • index_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • index_time_in_millis number

              Time unit for milliseconds

            • index_total number Required
            • index_failed number Required
            • types object
              Hide types attribute Show types attribute object
              • * object
            • write_load number
          • indices object Additional properties
          • merges object
            Hide merges attributes Show merges attributes object
            • current number Required
            • current_docs number Required
            • current_size string
            • current_size_in_bytes number Required
            • total number Required
            • total_auto_throttle string
            • total_auto_throttle_in_bytes number Required
            • total_docs number Required
            • total_size string
            • total_size_in_bytes number Required
            • total_stopped_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_stopped_time_in_millis number

              Time unit for milliseconds

            • total_throttled_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_throttled_time_in_millis number

              Time unit for milliseconds

            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

          • query_cache object
            Hide query_cache attributes Show query_cache attributes object
            • cache_count number Required

              Total number of entries added to the query cache across all shards assigned to selected nodes. This number includes current and evicted entries.

            • cache_size number Required

              Total number of entries currently in the query cache across all shards assigned to selected nodes.

            • evictions number Required

              Total number of query cache evictions across all shards assigned to selected nodes.

            • hit_count number Required

              Total count of query cache hits across all shards assigned to selected nodes.

            • memory_size number | string

            • memory_size_in_bytes number Required

              Total amount, in bytes, of memory used for the query cache across all shards assigned to selected nodes.

            • miss_count number Required

              Total count of query cache misses across all shards assigned to selected nodes.

            • total_count number Required

              Total count of hits and misses in the query cache across all shards assigned to selected nodes.

          • recovery object
            Hide recovery attributes Show recovery attributes object
            • current_as_source number Required
            • current_as_target number Required
            • throttle_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • throttle_time_in_millis number

              Time unit for milliseconds

          • refresh object
            Hide refresh attributes Show refresh attributes object
            • external_total number Required
            • external_total_time_in_millis number

              Time unit for milliseconds

            • listeners number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

          • request_cache object
            Hide request_cache attributes Show request_cache attributes object
            • evictions number Required
            • hit_count number Required
            • memory_size string
            • memory_size_in_bytes number Required
            • miss_count number Required
          • segments object
            Hide segments attributes Show segments attributes object
            • count number Required

              Total number of segments across all shards assigned to selected nodes.

            • doc_values_memory number | string

            • doc_values_memory_in_bytes number Required

              Total amount, in bytes, of memory used for doc values across all shards assigned to selected nodes.

            • file_sizes object Required

              This object is not populated by the cluster stats API. To get information on segment files, use the node stats API.

              Hide file_sizes attribute Show file_sizes attribute object
              • * object Additional properties
            • fixed_bit_set number | string

            • fixed_bit_set_memory_in_bytes number Required

              Total amount of memory, in bytes, used by fixed bit sets across all shards assigned to selected nodes.

            • index_writer_memory number | string

            • index_writer_max_memory_in_bytes number
            • index_writer_memory_in_bytes number Required

              Total amount, in bytes, of memory used by all index writers across all shards assigned to selected nodes.

            • max_unsafe_auto_id_timestamp number Required

              Unix timestamp, in milliseconds, of the most recently retried indexing request.

            • memory number | string

            • memory_in_bytes number Required

              Total amount, in bytes, of memory used for segments across all shards assigned to selected nodes.

            • norms_memory number | string

            • norms_memory_in_bytes number Required

              Total amount, in bytes, of memory used for normalization factors across all shards assigned to selected nodes.

            • points_memory number | string

            • points_memory_in_bytes number Required

              Total amount, in bytes, of memory used for points across all shards assigned to selected nodes.

            • stored_memory number | string

            • stored_fields_memory_in_bytes number Required

              Total amount, in bytes, of memory used for stored fields across all shards assigned to selected nodes.

            • terms_memory_in_bytes number Required

              Total amount, in bytes, of memory used for terms across all shards assigned to selected nodes.

            • terms_memory number | string

            • term_vectory_memory number | string

            • term_vectors_memory_in_bytes number Required

              Total amount, in bytes, of memory used for term vectors across all shards assigned to selected nodes.

            • version_map_memory number | string

            • version_map_memory_in_bytes number Required

              Total amount, in bytes, of memory used by all version maps across all shards assigned to selected nodes.

          • store object
            Hide store attributes Show store attributes object
            • size number | string

            • size_in_bytes number Required

              Total size, in bytes, of all shards assigned to selected nodes.

            • reserved number | string

            • reserved_in_bytes number Required

              A prediction, in bytes, of how much larger the shard stores will eventually grow due to ongoing peer recoveries, restoring snapshots, and similar activities.

            • total_data_set_size number | string

            • total_data_set_size_in_bytes number

              Total data set size, in bytes, of all shards assigned to selected nodes. This includes the size of shards not stored fully on the nodes, such as the cache for partially mounted indices.

          • translog object
            Hide translog attributes Show translog attributes object
            • earliest_last_modified_age number Required
            • operations number Required
            • size string
            • size_in_bytes number Required
            • uncommitted_operations number Required
            • uncommitted_size string
            • uncommitted_size_in_bytes number Required
          • warmer object
            Hide warmer attributes Show warmer attributes object
            • current number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

          • bulk object
            Hide bulk attributes Show bulk attributes object
            • total_operations number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

            • total_size number | string

            • total_size_in_bytes number Required
            • avg_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • avg_time_in_millis number

              Time unit for milliseconds

            • avg_size number | string

            • avg_size_in_bytes number Required
          • shard_stats object
            Hide shard_stats attribute Show shard_stats attribute object
            • total_count number Required
        • uuid string
        • health string

          Values are green, GREEN, yellow, YELLOW, red, or RED.

        • status string

          Values are open or close.

    • _shards object Required
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number Required
        • status string
      • skipped number
    • _all object Required Additional properties
      Hide _all attributes Show _all attributes object
      • primaries object
        Hide primaries attributes Show primaries attributes object
        • completion object
          Hide completion attributes Show completion attributes object
          • size_in_bytes number Required

            Total amount, in bytes, of memory used for completion across all shards assigned to selected nodes.

          • size number | string

          • fields object
            Hide fields attribute Show fields attribute object
            • * object Additional properties
              Hide * attributes Show * attributes object
              • size
              • size_in_bytes number Required
        • docs object
          Hide docs attributes Show docs attributes object
          • count number Required

            Total number of non-deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments and may include documents from nested fields.

          • deleted number

            Total number of deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments. Elasticsearch reclaims the disk space of deleted Lucene documents when a segment is merged.

        • fielddata object
          Hide fielddata attributes Show fielddata attributes object
          • evictions number
          • memory_size number | string

          • memory_size_in_bytes number Required
          • fields object
            Hide fields attribute Show fields attribute object
            • * object Additional properties
              Hide * attributes Show * attributes object
              • memory_size
              • memory_size_in_bytes number Required
        • flush object
          Hide flush attributes Show flush attributes object
          • periodic number Required
          • total number Required
          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

        • get object
          Hide get attributes Show get attributes object
          • current number Required
          • exists_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • exists_time_in_millis number

            Time unit for milliseconds

          • exists_total number Required
          • missing_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • missing_time_in_millis number

            Time unit for milliseconds

          • missing_total number Required
          • time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • time_in_millis number

            Time unit for milliseconds

          • total number Required
        • indexing object
          Hide indexing attributes Show indexing attributes object
          • index_current number Required
          • delete_current number Required
          • delete_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • delete_time_in_millis number

            Time unit for milliseconds

          • delete_total number Required
          • is_throttled boolean Required
          • noop_update_total number Required
          • throttle_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • throttle_time_in_millis number

            Time unit for milliseconds

          • index_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • index_time_in_millis number

            Time unit for milliseconds

          • index_total number Required
          • index_failed number Required
          • types object
            Hide types attribute Show types attribute object
            • * object
          • write_load number
        • indices object Additional properties
        • merges object
          Hide merges attributes Show merges attributes object
          • current number Required
          • current_docs number Required
          • current_size string
          • current_size_in_bytes number Required
          • total number Required
          • total_auto_throttle string
          • total_auto_throttle_in_bytes number Required
          • total_docs number Required
          • total_size string
          • total_size_in_bytes number Required
          • total_stopped_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_stopped_time_in_millis number

            Time unit for milliseconds

          • total_throttled_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_throttled_time_in_millis number

            Time unit for milliseconds

          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

        • query_cache object
          Hide query_cache attributes Show query_cache attributes object
          • cache_count number Required

            Total number of entries added to the query cache across all shards assigned to selected nodes. This number includes current and evicted entries.

          • cache_size number Required

            Total number of entries currently in the query cache across all shards assigned to selected nodes.

          • evictions number Required

            Total number of query cache evictions across all shards assigned to selected nodes.

          • hit_count number Required

            Total count of query cache hits across all shards assigned to selected nodes.

          • memory_size number | string

          • memory_size_in_bytes number Required

            Total amount, in bytes, of memory used for the query cache across all shards assigned to selected nodes.

          • miss_count number Required

            Total count of query cache misses across all shards assigned to selected nodes.

          • total_count number Required

            Total count of hits and misses in the query cache across all shards assigned to selected nodes.

        • recovery object
          Hide recovery attributes Show recovery attributes object
          • current_as_source number Required
          • current_as_target number Required
          • throttle_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • throttle_time_in_millis number

            Time unit for milliseconds

        • refresh object
          Hide refresh attributes Show refresh attributes object
          • external_total number Required
          • external_total_time_in_millis number

            Time unit for milliseconds

          • listeners number Required
          • total number Required
          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

        • request_cache object
          Hide request_cache attributes Show request_cache attributes object
          • evictions number Required
          • hit_count number Required
          • memory_size string
          • memory_size_in_bytes number Required
          • miss_count number Required
        • segments object
          Hide segments attributes Show segments attributes object
          • count number Required

            Total number of segments across all shards assigned to selected nodes.

          • doc_values_memory number | string

          • doc_values_memory_in_bytes number Required

            Total amount, in bytes, of memory used for doc values across all shards assigned to selected nodes.

          • file_sizes object Required

            This object is not populated by the cluster stats API. To get information on segment files, use the node stats API.

            Hide file_sizes attribute Show file_sizes attribute object
            • * object Additional properties
              Hide * attributes Show * attributes object
              • description string Required
              • size_in_bytes number Required
              • min_size_in_bytes number
              • max_size_in_bytes number
              • average_size_in_bytes number
              • count number
          • fixed_bit_set number | string

          • fixed_bit_set_memory_in_bytes number Required

            Total amount of memory, in bytes, used by fixed bit sets across all shards assigned to selected nodes.

          • index_writer_memory number | string

          • index_writer_max_memory_in_bytes number
          • index_writer_memory_in_bytes number Required

            Total amount, in bytes, of memory used by all index writers across all shards assigned to selected nodes.

          • max_unsafe_auto_id_timestamp number Required

            Unix timestamp, in milliseconds, of the most recently retried indexing request.

          • memory number | string

          • memory_in_bytes number Required

            Total amount, in bytes, of memory used for segments across all shards assigned to selected nodes.

          • norms_memory number | string

          • norms_memory_in_bytes number Required

            Total amount, in bytes, of memory used for normalization factors across all shards assigned to selected nodes.

          • points_memory number | string

          • points_memory_in_bytes number Required

            Total amount, in bytes, of memory used for points across all shards assigned to selected nodes.

          • stored_memory number | string

          • stored_fields_memory_in_bytes number Required

            Total amount, in bytes, of memory used for stored fields across all shards assigned to selected nodes.

          • terms_memory_in_bytes number Required

            Total amount, in bytes, of memory used for terms across all shards assigned to selected nodes.

          • terms_memory number | string

          • term_vectory_memory number | string

          • term_vectors_memory_in_bytes number Required

            Total amount, in bytes, of memory used for term vectors across all shards assigned to selected nodes.

          • version_map_memory number | string

          • version_map_memory_in_bytes number Required

            Total amount, in bytes, of memory used by all version maps across all shards assigned to selected nodes.

        • store object
          Hide store attributes Show store attributes object
          • size number | string

          • size_in_bytes number Required

            Total size, in bytes, of all shards assigned to selected nodes.

          • reserved number | string

          • reserved_in_bytes number Required

            A prediction, in bytes, of how much larger the shard stores will eventually grow due to ongoing peer recoveries, restoring snapshots, and similar activities.

          • total_data_set_size number | string

          • total_data_set_size_in_bytes number

            Total data set size, in bytes, of all shards assigned to selected nodes. This includes the size of shards not stored fully on the nodes, such as the cache for partially mounted indices.

        • translog object
          Hide translog attributes Show translog attributes object
          • earliest_last_modified_age number Required
          • operations number Required
          • size string
          • size_in_bytes number Required
          • uncommitted_operations number Required
          • uncommitted_size string
          • uncommitted_size_in_bytes number Required
        • warmer object
          Hide warmer attributes Show warmer attributes object
          • current number Required
          • total number Required
          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

        • bulk object
          Hide bulk attributes Show bulk attributes object
          • total_operations number Required
          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

          • total_size number | string

          • total_size_in_bytes number Required
          • avg_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • avg_time_in_millis number

            Time unit for milliseconds

          • avg_size number | string

          • avg_size_in_bytes number Required
        • shard_stats object
          Hide shard_stats attribute Show shard_stats attribute object
          • total_count number Required
      • shards object
        Hide shards attribute Show shards attribute object
        • * array[object] Additional properties
          Hide * attributes Show * attributes object
          • commit object
            Hide commit attributes Show commit attributes object
            • generation number Required
            • id string Required
            • num_docs number Required
            • user_data object Required
          • completion object
            Hide completion attributes Show completion attributes object
            • size_in_bytes number Required

              Total amount, in bytes, of memory used for completion across all shards assigned to selected nodes.

            • size
            • fields object
          • docs object
            Hide docs attributes Show docs attributes object
            • count number Required

              Total number of non-deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments and may include documents from nested fields.

            • deleted number

              Total number of deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments. Elasticsearch reclaims the disk space of deleted Lucene documents when a segment is merged.

          • fielddata object
            Hide fielddata attributes Show fielddata attributes object
            • evictions number
            • memory_size
            • memory_size_in_bytes number Required
            • fields object
          • flush object
            Hide flush attributes Show flush attributes object
            • periodic number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis
          • get object
            Hide get attributes Show get attributes object
            • current number Required
            • exists_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • exists_time_in_millis
            • exists_total number Required
            • missing_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • missing_time_in_millis
            • missing_total number Required
            • time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • time_in_millis
            • total number Required
          • indexing object
            Hide indexing attributes Show indexing attributes object
            • index_current number Required
            • delete_current number Required
            • delete_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • delete_time_in_millis
            • delete_total number Required
            • is_throttled boolean Required
            • noop_update_total number Required
            • throttle_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • throttle_time_in_millis
            • index_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • index_time_in_millis
            • index_total number Required
            • index_failed number Required
            • types object
            • write_load number
          • mappings object
            Hide mappings attributes Show mappings attributes object
            • total_count number Required
            • total_estimated_overhead
            • total_estimated_overhead_in_bytes number Required
          • merges object
            Hide merges attributes Show merges attributes object
            • current number Required
            • current_docs number Required
            • current_size string
            • current_size_in_bytes number Required
            • total number Required
            • total_auto_throttle string
            • total_auto_throttle_in_bytes number Required
            • total_docs number Required
            • total_size string
            • total_size_in_bytes number Required
            • total_stopped_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_stopped_time_in_millis
            • total_throttled_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_throttled_time_in_millis
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis
          • shard_path object
            Hide shard_path attributes Show shard_path attributes object
            • data_path string Required
            • is_custom_data_path boolean Required
            • state_path string Required
          • query_cache object
            Hide query_cache attributes Show query_cache attributes object
            • cache_count number Required
            • cache_size number Required
            • evictions number Required
            • hit_count number Required
            • memory_size_in_bytes number Required
            • miss_count number Required
            • total_count number Required
          • recovery object
            Hide recovery attributes Show recovery attributes object
            • current_as_source number Required
            • current_as_target number Required
            • throttle_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • throttle_time_in_millis
          • refresh object
            Hide refresh attributes Show refresh attributes object
            • external_total number Required
            • external_total_time_in_millis
            • listeners number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis
          • request_cache object
            Hide request_cache attributes Show request_cache attributes object
            • evictions number Required
            • hit_count number Required
            • memory_size string
            • memory_size_in_bytes number Required
            • miss_count number Required
          • retention_leases object
            Hide retention_leases attributes Show retention_leases attributes object
            • primary_term number Required
            • version number Required
            • leases array[object] Required
          • routing object
            Hide routing attributes Show routing attributes object
            • node string Required
            • primary boolean Required
            • relocating_node
            • state string Required

              Values are UNASSIGNED, INITIALIZING, STARTED, or RELOCATING.

          • segments object
            Hide segments attributes Show segments attributes object
            • count number Required

              Total number of segments across all shards assigned to selected nodes.

            • doc_values_memory
            • doc_values_memory_in_bytes number Required

              Total amount, in bytes, of memory used for doc values across all shards assigned to selected nodes.

            • file_sizes object Required

              This object is not populated by the cluster stats API. To get information on segment files, use the node stats API.

            • fixed_bit_set
            • fixed_bit_set_memory_in_bytes number Required

              Total amount of memory, in bytes, used by fixed bit sets across all shards assigned to selected nodes.

            • index_writer_memory
            • index_writer_max_memory_in_bytes number
            • index_writer_memory_in_bytes number Required

              Total amount, in bytes, of memory used by all index writers across all shards assigned to selected nodes.

            • max_unsafe_auto_id_timestamp number Required

              Unix timestamp, in milliseconds, of the most recently retried indexing request.

            • memory
            • memory_in_bytes number Required

              Total amount, in bytes, of memory used for segments across all shards assigned to selected nodes.

            • norms_memory
            • norms_memory_in_bytes number Required

              Total amount, in bytes, of memory used for normalization factors across all shards assigned to selected nodes.

            • points_memory
            • points_memory_in_bytes number Required

              Total amount, in bytes, of memory used for points across all shards assigned to selected nodes.

            • stored_memory
            • stored_fields_memory_in_bytes number Required

              Total amount, in bytes, of memory used for stored fields across all shards assigned to selected nodes.

            • terms_memory_in_bytes number Required

              Total amount, in bytes, of memory used for terms across all shards assigned to selected nodes.

            • terms_memory
            • term_vectory_memory
            • term_vectors_memory_in_bytes number Required

              Total amount, in bytes, of memory used for term vectors across all shards assigned to selected nodes.

            • version_map_memory
            • version_map_memory_in_bytes number Required

              Total amount, in bytes, of memory used by all version maps across all shards assigned to selected nodes.

          • seq_no object
            Hide seq_no attributes Show seq_no attributes object
            • global_checkpoint number Required
            • local_checkpoint number Required
            • max_seq_no number Required
          • store object
            Hide store attributes Show store attributes object
            • size
            • size_in_bytes number Required

              Total size, in bytes, of all shards assigned to selected nodes.

            • reserved
            • reserved_in_bytes number Required

              A prediction, in bytes, of how much larger the shard stores will eventually grow due to ongoing peer recoveries, restoring snapshots, and similar activities.

            • total_data_set_size
            • total_data_set_size_in_bytes number

              Total data set size, in bytes, of all shards assigned to selected nodes. This includes the size of shards not stored fully on the nodes, such as the cache for partially mounted indices.

          • translog object
            Hide translog attributes Show translog attributes object
            • earliest_last_modified_age number Required
            • operations number Required
            • size string
            • size_in_bytes number Required
            • uncommitted_operations number Required
            • uncommitted_size string
            • uncommitted_size_in_bytes number Required
          • warmer object
            Hide warmer attributes Show warmer attributes object
            • current number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis
          • bulk object
            Hide bulk attributes Show bulk attributes object
            • total_operations number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis
            • total_size
            • total_size_in_bytes number Required
            • avg_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • avg_time_in_millis
            • avg_size
            • avg_size_in_bytes number Required
          • shards object Generally available; Added in 7.15.0
            Hide shards attribute Show shards attribute object
            • * object Additional properties
          • shard_stats object
            Hide shard_stats attribute Show shard_stats attribute object
            • total_count number Required
          • indices object Additional properties
      • total object
        Hide total attributes Show total attributes object
        • completion object
          Hide completion attributes Show completion attributes object
          • size_in_bytes number Required

            Total amount, in bytes, of memory used for completion across all shards assigned to selected nodes.

          • size number | string

          • fields object
            Hide fields attribute Show fields attribute object
            • * object Additional properties
              Hide * attributes Show * attributes object
              • size
              • size_in_bytes number Required
        • docs object
          Hide docs attributes Show docs attributes object
          • count number Required

            Total number of non-deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments and may include documents from nested fields.

          • deleted number

            Total number of deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments. Elasticsearch reclaims the disk space of deleted Lucene documents when a segment is merged.

        • fielddata object
          Hide fielddata attributes Show fielddata attributes object
          • evictions number
          • memory_size number | string

          • memory_size_in_bytes number Required
          • fields object
            Hide fields attribute Show fields attribute object
            • * object Additional properties
              Hide * attributes Show * attributes object
              • memory_size
              • memory_size_in_bytes number Required
        • flush object
          Hide flush attributes Show flush attributes object
          • periodic number Required
          • total number Required
          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

        • get object
          Hide get attributes Show get attributes object
          • current number Required
          • exists_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • exists_time_in_millis number

            Time unit for milliseconds

          • exists_total number Required
          • missing_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • missing_time_in_millis number

            Time unit for milliseconds

          • missing_total number Required
          • time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • time_in_millis number

            Time unit for milliseconds

          • total number Required
        • indexing object
          Hide indexing attributes Show indexing attributes object
          • index_current number Required
          • delete_current number Required
          • delete_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • delete_time_in_millis number

            Time unit for milliseconds

          • delete_total number Required
          • is_throttled boolean Required
          • noop_update_total number Required
          • throttle_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • throttle_time_in_millis number

            Time unit for milliseconds

          • index_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • index_time_in_millis number

            Time unit for milliseconds

          • index_total number Required
          • index_failed number Required
          • types object
            Hide types attribute Show types attribute object
            • * object
          • write_load number
        • indices object Additional properties
        • merges object
          Hide merges attributes Show merges attributes object
          • current number Required
          • current_docs number Required
          • current_size string
          • current_size_in_bytes number Required
          • total number Required
          • total_auto_throttle string
          • total_auto_throttle_in_bytes number Required
          • total_docs number Required
          • total_size string
          • total_size_in_bytes number Required
          • total_stopped_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_stopped_time_in_millis number

            Time unit for milliseconds

          • total_throttled_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_throttled_time_in_millis number

            Time unit for milliseconds

          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

        • query_cache object
          Hide query_cache attributes Show query_cache attributes object
          • cache_count number Required

            Total number of entries added to the query cache across all shards assigned to selected nodes. This number includes current and evicted entries.

          • cache_size number Required

            Total number of entries currently in the query cache across all shards assigned to selected nodes.

          • evictions number Required

            Total number of query cache evictions across all shards assigned to selected nodes.

          • hit_count number Required

            Total count of query cache hits across all shards assigned to selected nodes.

          • memory_size number | string

          • memory_size_in_bytes number Required

            Total amount, in bytes, of memory used for the query cache across all shards assigned to selected nodes.

          • miss_count number Required

            Total count of query cache misses across all shards assigned to selected nodes.

          • total_count number Required

            Total count of hits and misses in the query cache across all shards assigned to selected nodes.

        • recovery object
          Hide recovery attributes Show recovery attributes object
          • current_as_source number Required
          • current_as_target number Required
          • throttle_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • throttle_time_in_millis number

            Time unit for milliseconds

        • refresh object
          Hide refresh attributes Show refresh attributes object
          • external_total number Required
          • external_total_time_in_millis number

            Time unit for milliseconds

          • listeners number Required
          • total number Required
          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

        • request_cache object
          Hide request_cache attributes Show request_cache attributes object
          • evictions number Required
          • hit_count number Required
          • memory_size string
          • memory_size_in_bytes number Required
          • miss_count number Required
        • segments object
          Hide segments attributes Show segments attributes object
          • count number Required

            Total number of segments across all shards assigned to selected nodes.

          • doc_values_memory number | string

          • doc_values_memory_in_bytes number Required

            Total amount, in bytes, of memory used for doc values across all shards assigned to selected nodes.

          • file_sizes object Required

            This object is not populated by the cluster stats API. To get information on segment files, use the node stats API.

            Hide file_sizes attribute Show file_sizes attribute object
            • * object Additional properties
              Hide * attributes Show * attributes object
              • description string Required
              • size_in_bytes number Required
              • min_size_in_bytes number
              • max_size_in_bytes number
              • average_size_in_bytes number
              • count number
          • fixed_bit_set number | string

          • fixed_bit_set_memory_in_bytes number Required

            Total amount of memory, in bytes, used by fixed bit sets across all shards assigned to selected nodes.

          • index_writer_memory number | string

          • index_writer_max_memory_in_bytes number
          • index_writer_memory_in_bytes number Required

            Total amount, in bytes, of memory used by all index writers across all shards assigned to selected nodes.

          • max_unsafe_auto_id_timestamp number Required

            Unix timestamp, in milliseconds, of the most recently retried indexing request.

          • memory number | string

          • memory_in_bytes number Required

            Total amount, in bytes, of memory used for segments across all shards assigned to selected nodes.

          • norms_memory number | string

          • norms_memory_in_bytes number Required

            Total amount, in bytes, of memory used for normalization factors across all shards assigned to selected nodes.

          • points_memory number | string

          • points_memory_in_bytes number Required

            Total amount, in bytes, of memory used for points across all shards assigned to selected nodes.

          • stored_memory number | string

          • stored_fields_memory_in_bytes number Required

            Total amount, in bytes, of memory used for stored fields across all shards assigned to selected nodes.

          • terms_memory_in_bytes number Required

            Total amount, in bytes, of memory used for terms across all shards assigned to selected nodes.

          • terms_memory number | string

          • term_vectory_memory number | string

          • term_vectors_memory_in_bytes number Required

            Total amount, in bytes, of memory used for term vectors across all shards assigned to selected nodes.

          • version_map_memory number | string

          • version_map_memory_in_bytes number Required

            Total amount, in bytes, of memory used by all version maps across all shards assigned to selected nodes.

        • store object
          Hide store attributes Show store attributes object
          • size number | string

          • size_in_bytes number Required

            Total size, in bytes, of all shards assigned to selected nodes.

          • reserved number | string

          • reserved_in_bytes number Required

            A prediction, in bytes, of how much larger the shard stores will eventually grow due to ongoing peer recoveries, restoring snapshots, and similar activities.

          • total_data_set_size number | string

          • total_data_set_size_in_bytes number

            Total data set size, in bytes, of all shards assigned to selected nodes. This includes the size of shards not stored fully on the nodes, such as the cache for partially mounted indices.

        • translog object
          Hide translog attributes Show translog attributes object
          • earliest_last_modified_age number Required
          • operations number Required
          • size string
          • size_in_bytes number Required
          • uncommitted_operations number Required
          • uncommitted_size string
          • uncommitted_size_in_bytes number Required
        • warmer object
          Hide warmer attributes Show warmer attributes object
          • current number Required
          • total number Required
          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

        • bulk object
          Hide bulk attributes Show bulk attributes object
          • total_operations number Required
          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

          • total_size number | string

          • total_size_in_bytes number Required
          • avg_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • avg_time_in_millis number

            Time unit for milliseconds

          • avg_size number | string

          • avg_size_in_bytes number Required
        • shard_stats object
          Hide shard_stats attribute Show shard_stats attribute object
          • total_count number Required
      • uuid string
      • health string

        Values are green, GREEN, yellow, YELLOW, red, or RED.

      • status string

        Values are open or close.

GET /_stats
curl \
 --request GET 'http://api.example.com/_stats' \
 --header "Authorization: $API_KEY"




Get index statistics Generally available; Added in 1.3.0

GET /{index}/_stats

For data streams, the API retrieves statistics for the stream's backing indices.

By default, the returned statistics are index-level with primaries and total aggregations. primaries are the values for only the primary shards. total are the accumulated values for both primary and replica shards.

To get shard-level statistics, set the level parameter to shards.

NOTE: When moving to another node, the shard-level statistics for a shard are cleared. Although the shard is no longer part of the node, that node retains any node-level statistics to which the shard contributed.

Required authorization

  • Index privileges: monitor

Path parameters

  • index string | array[string] Required

    A comma-separated list of index names; use _all or empty string to perform the operation on all indices

Query parameters

  • completion_fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in fielddata and suggest statistics.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • fielddata_fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in fielddata statistics.

  • fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in the statistics.

  • forbid_closed_indices boolean

    If true, statistics are not collected from closed indices.

  • groups string | array[string]

    Comma-separated list of search groups to include in the search statistics.

  • include_segment_file_sizes boolean

    If true, the call reports the aggregated disk usage of each one of the Lucene index files (only applies if segment stats are requested).

  • include_unloaded_segments boolean

    If true, the response includes information from segments that are not loaded into memory.

  • level string

    Indicates whether statistics are aggregated at the cluster, index, or shard level.

    Values are cluster, indices, or shards.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • indices object
      Hide indices attribute Show indices attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • primaries object
          Hide primaries attributes Show primaries attributes object
          • completion object
            Hide completion attributes Show completion attributes object
            • size_in_bytes number Required

              Total amount, in bytes, of memory used for completion across all shards assigned to selected nodes.

            • size number | string

            • fields object
              Hide fields attribute Show fields attribute object
              • * object Additional properties
          • docs object
            Hide docs attributes Show docs attributes object
            • count number Required

              Total number of non-deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments and may include documents from nested fields.

            • deleted number

              Total number of deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments. Elasticsearch reclaims the disk space of deleted Lucene documents when a segment is merged.

          • fielddata object
            Hide fielddata attributes Show fielddata attributes object
            • evictions number
            • memory_size number | string

            • memory_size_in_bytes number Required
            • fields object
              Hide fields attribute Show fields attribute object
              • * object Additional properties
          • flush object
            Hide flush attributes Show flush attributes object
            • periodic number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

          • get object
            Hide get attributes Show get attributes object
            • current number Required
            • exists_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • exists_time_in_millis number

              Time unit for milliseconds

            • exists_total number Required
            • missing_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • missing_time_in_millis number

              Time unit for milliseconds

            • missing_total number Required
            • time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • time_in_millis number

              Time unit for milliseconds

            • total number Required
          • indexing object
            Hide indexing attributes Show indexing attributes object
            • index_current number Required
            • delete_current number Required
            • delete_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • delete_time_in_millis number

              Time unit for milliseconds

            • delete_total number Required
            • is_throttled boolean Required
            • noop_update_total number Required
            • throttle_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • throttle_time_in_millis number

              Time unit for milliseconds

            • index_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • index_time_in_millis number

              Time unit for milliseconds

            • index_total number Required
            • index_failed number Required
            • types object
              Hide types attribute Show types attribute object
              • * object
            • write_load number
          • indices object Additional properties
          • merges object
            Hide merges attributes Show merges attributes object
            • current number Required
            • current_docs number Required
            • current_size string
            • current_size_in_bytes number Required
            • total number Required
            • total_auto_throttle string
            • total_auto_throttle_in_bytes number Required
            • total_docs number Required
            • total_size string
            • total_size_in_bytes number Required
            • total_stopped_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_stopped_time_in_millis number

              Time unit for milliseconds

            • total_throttled_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_throttled_time_in_millis number

              Time unit for milliseconds

            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

          • query_cache object
            Hide query_cache attributes Show query_cache attributes object
            • cache_count number Required

              Total number of entries added to the query cache across all shards assigned to selected nodes. This number includes current and evicted entries.

            • cache_size number Required

              Total number of entries currently in the query cache across all shards assigned to selected nodes.

            • evictions number Required

              Total number of query cache evictions across all shards assigned to selected nodes.

            • hit_count number Required

              Total count of query cache hits across all shards assigned to selected nodes.

            • memory_size number | string

            • memory_size_in_bytes number Required

              Total amount, in bytes, of memory used for the query cache across all shards assigned to selected nodes.

            • miss_count number Required

              Total count of query cache misses across all shards assigned to selected nodes.

            • total_count number Required

              Total count of hits and misses in the query cache across all shards assigned to selected nodes.

          • recovery object
            Hide recovery attributes Show recovery attributes object
            • current_as_source number Required
            • current_as_target number Required
            • throttle_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • throttle_time_in_millis number

              Time unit for milliseconds

          • refresh object
            Hide refresh attributes Show refresh attributes object
            • external_total number Required
            • external_total_time_in_millis number

              Time unit for milliseconds

            • listeners number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

          • request_cache object
            Hide request_cache attributes Show request_cache attributes object
            • evictions number Required
            • hit_count number Required
            • memory_size string
            • memory_size_in_bytes number Required
            • miss_count number Required
          • segments object
            Hide segments attributes Show segments attributes object
            • count number Required

              Total number of segments across all shards assigned to selected nodes.

            • doc_values_memory number | string

            • doc_values_memory_in_bytes number Required

              Total amount, in bytes, of memory used for doc values across all shards assigned to selected nodes.

            • file_sizes object Required

              This object is not populated by the cluster stats API. To get information on segment files, use the node stats API.

              Hide file_sizes attribute Show file_sizes attribute object
              • * object Additional properties
            • fixed_bit_set number | string

            • fixed_bit_set_memory_in_bytes number Required

              Total amount of memory, in bytes, used by fixed bit sets across all shards assigned to selected nodes.

            • index_writer_memory number | string

            • index_writer_max_memory_in_bytes number
            • index_writer_memory_in_bytes number Required

              Total amount, in bytes, of memory used by all index writers across all shards assigned to selected nodes.

            • max_unsafe_auto_id_timestamp number Required

              Unix timestamp, in milliseconds, of the most recently retried indexing request.

            • memory number | string

            • memory_in_bytes number Required

              Total amount, in bytes, of memory used for segments across all shards assigned to selected nodes.

            • norms_memory number | string

            • norms_memory_in_bytes number Required

              Total amount, in bytes, of memory used for normalization factors across all shards assigned to selected nodes.

            • points_memory number | string

            • points_memory_in_bytes number Required

              Total amount, in bytes, of memory used for points across all shards assigned to selected nodes.

            • stored_memory number | string

            • stored_fields_memory_in_bytes number Required

              Total amount, in bytes, of memory used for stored fields across all shards assigned to selected nodes.

            • terms_memory_in_bytes number Required

              Total amount, in bytes, of memory used for terms across all shards assigned to selected nodes.

            • terms_memory number | string

            • term_vectory_memory number | string

            • term_vectors_memory_in_bytes number Required

              Total amount, in bytes, of memory used for term vectors across all shards assigned to selected nodes.

            • version_map_memory number | string

            • version_map_memory_in_bytes number Required

              Total amount, in bytes, of memory used by all version maps across all shards assigned to selected nodes.

          • store object
            Hide store attributes Show store attributes object
            • size number | string

            • size_in_bytes number Required

              Total size, in bytes, of all shards assigned to selected nodes.

            • reserved number | string

            • reserved_in_bytes number Required

              A prediction, in bytes, of how much larger the shard stores will eventually grow due to ongoing peer recoveries, restoring snapshots, and similar activities.

            • total_data_set_size number | string

            • total_data_set_size_in_bytes number

              Total data set size, in bytes, of all shards assigned to selected nodes. This includes the size of shards not stored fully on the nodes, such as the cache for partially mounted indices.

          • translog object
            Hide translog attributes Show translog attributes object
            • earliest_last_modified_age number Required
            • operations number Required
            • size string
            • size_in_bytes number Required
            • uncommitted_operations number Required
            • uncommitted_size string
            • uncommitted_size_in_bytes number Required
          • warmer object
            Hide warmer attributes Show warmer attributes object
            • current number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

          • bulk object
            Hide bulk attributes Show bulk attributes object
            • total_operations number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

            • total_size number | string

            • total_size_in_bytes number Required
            • avg_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • avg_time_in_millis number

              Time unit for milliseconds

            • avg_size number | string

            • avg_size_in_bytes number Required
          • shard_stats object
            Hide shard_stats attribute Show shard_stats attribute object
            • total_count number Required
        • shards object
          Hide shards attribute Show shards attribute object
          • * array[object] Additional properties
            Hide * attributes Show * attributes object
            • commit object
            • completion object
            • docs object
            • fielddata object
            • flush object
            • get object
            • indexing object
            • mappings object
            • merges object
            • shard_path object
            • query_cache object
            • recovery object
            • refresh object
            • request_cache object
            • retention_leases object
            • routing object
            • segments object
            • seq_no object
            • store object
            • translog object
            • warmer object
            • bulk object
            • shards object Generally available; Added in 7.15.0
            • shard_stats object
            • indices object Additional properties
        • total object
          Hide total attributes Show total attributes object
          • completion object
            Hide completion attributes Show completion attributes object
            • size_in_bytes number Required

              Total amount, in bytes, of memory used for completion across all shards assigned to selected nodes.

            • size number | string

            • fields object
              Hide fields attribute Show fields attribute object
              • * object Additional properties
          • docs object
            Hide docs attributes Show docs attributes object
            • count number Required

              Total number of non-deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments and may include documents from nested fields.

            • deleted number

              Total number of deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments. Elasticsearch reclaims the disk space of deleted Lucene documents when a segment is merged.

          • fielddata object
            Hide fielddata attributes Show fielddata attributes object
            • evictions number
            • memory_size number | string

            • memory_size_in_bytes number Required
            • fields object
              Hide fields attribute Show fields attribute object
              • * object Additional properties
          • flush object
            Hide flush attributes Show flush attributes object
            • periodic number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

          • get object
            Hide get attributes Show get attributes object
            • current number Required
            • exists_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • exists_time_in_millis number

              Time unit for milliseconds

            • exists_total number Required
            • missing_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • missing_time_in_millis number

              Time unit for milliseconds

            • missing_total number Required
            • time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • time_in_millis number

              Time unit for milliseconds

            • total number Required
          • indexing object
            Hide indexing attributes Show indexing attributes object
            • index_current number Required
            • delete_current number Required
            • delete_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • delete_time_in_millis number

              Time unit for milliseconds

            • delete_total number Required
            • is_throttled boolean Required
            • noop_update_total number Required
            • throttle_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • throttle_time_in_millis number

              Time unit for milliseconds

            • index_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • index_time_in_millis number

              Time unit for milliseconds

            • index_total number Required
            • index_failed number Required
            • types object
              Hide types attribute Show types attribute object
              • * object
            • write_load number
          • indices object Additional properties
          • merges object
            Hide merges attributes Show merges attributes object
            • current number Required
            • current_docs number Required
            • current_size string
            • current_size_in_bytes number Required
            • total number Required
            • total_auto_throttle string
            • total_auto_throttle_in_bytes number Required
            • total_docs number Required
            • total_size string
            • total_size_in_bytes number Required
            • total_stopped_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_stopped_time_in_millis number

              Time unit for milliseconds

            • total_throttled_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_throttled_time_in_millis number

              Time unit for milliseconds

            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

          • query_cache object
            Hide query_cache attributes Show query_cache attributes object
            • cache_count number Required

              Total number of entries added to the query cache across all shards assigned to selected nodes. This number includes current and evicted entries.

            • cache_size number Required

              Total number of entries currently in the query cache across all shards assigned to selected nodes.

            • evictions number Required

              Total number of query cache evictions across all shards assigned to selected nodes.

            • hit_count number Required

              Total count of query cache hits across all shards assigned to selected nodes.

            • memory_size number | string

            • memory_size_in_bytes number Required

              Total amount, in bytes, of memory used for the query cache across all shards assigned to selected nodes.

            • miss_count number Required

              Total count of query cache misses across all shards assigned to selected nodes.

            • total_count number Required

              Total count of hits and misses in the query cache across all shards assigned to selected nodes.

          • recovery object
            Hide recovery attributes Show recovery attributes object
            • current_as_source number Required
            • current_as_target number Required
            • throttle_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • throttle_time_in_millis number

              Time unit for milliseconds

          • refresh object
            Hide refresh attributes Show refresh attributes object
            • external_total number Required
            • external_total_time_in_millis number

              Time unit for milliseconds

            • listeners number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

          • request_cache object
            Hide request_cache attributes Show request_cache attributes object
            • evictions number Required
            • hit_count number Required
            • memory_size string
            • memory_size_in_bytes number Required
            • miss_count number Required
          • segments object
            Hide segments attributes Show segments attributes object
            • count number Required

              Total number of segments across all shards assigned to selected nodes.

            • doc_values_memory number | string

            • doc_values_memory_in_bytes number Required

              Total amount, in bytes, of memory used for doc values across all shards assigned to selected nodes.

            • file_sizes object Required

              This object is not populated by the cluster stats API. To get information on segment files, use the node stats API.

              Hide file_sizes attribute Show file_sizes attribute object
              • * object Additional properties
            • fixed_bit_set number | string

            • fixed_bit_set_memory_in_bytes number Required

              Total amount of memory, in bytes, used by fixed bit sets across all shards assigned to selected nodes.

            • index_writer_memory number | string

            • index_writer_max_memory_in_bytes number
            • index_writer_memory_in_bytes number Required

              Total amount, in bytes, of memory used by all index writers across all shards assigned to selected nodes.

            • max_unsafe_auto_id_timestamp number Required

              Unix timestamp, in milliseconds, of the most recently retried indexing request.

            • memory number | string

            • memory_in_bytes number Required

              Total amount, in bytes, of memory used for segments across all shards assigned to selected nodes.

            • norms_memory number | string

            • norms_memory_in_bytes number Required

              Total amount, in bytes, of memory used for normalization factors across all shards assigned to selected nodes.

            • points_memory number | string

            • points_memory_in_bytes number Required

              Total amount, in bytes, of memory used for points across all shards assigned to selected nodes.

            • stored_memory number | string

            • stored_fields_memory_in_bytes number Required

              Total amount, in bytes, of memory used for stored fields across all shards assigned to selected nodes.

            • terms_memory_in_bytes number Required

              Total amount, in bytes, of memory used for terms across all shards assigned to selected nodes.

            • terms_memory number | string

            • term_vectory_memory number | string

            • term_vectors_memory_in_bytes number Required

              Total amount, in bytes, of memory used for term vectors across all shards assigned to selected nodes.

            • version_map_memory number | string

            • version_map_memory_in_bytes number Required

              Total amount, in bytes, of memory used by all version maps across all shards assigned to selected nodes.

          • store object
            Hide store attributes Show store attributes object
            • size number | string

            • size_in_bytes number Required

              Total size, in bytes, of all shards assigned to selected nodes.

            • reserved number | string

            • reserved_in_bytes number Required

              A prediction, in bytes, of how much larger the shard stores will eventually grow due to ongoing peer recoveries, restoring snapshots, and similar activities.

            • total_data_set_size number | string

            • total_data_set_size_in_bytes number

              Total data set size, in bytes, of all shards assigned to selected nodes. This includes the size of shards not stored fully on the nodes, such as the cache for partially mounted indices.

          • translog object
            Hide translog attributes Show translog attributes object
            • earliest_last_modified_age number Required
            • operations number Required
            • size string
            • size_in_bytes number Required
            • uncommitted_operations number Required
            • uncommitted_size string
            • uncommitted_size_in_bytes number Required
          • warmer object
            Hide warmer attributes Show warmer attributes object
            • current number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

          • bulk object
            Hide bulk attributes Show bulk attributes object
            • total_operations number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis number

              Time unit for milliseconds

            • total_size number | string

            • total_size_in_bytes number Required
            • avg_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • avg_time_in_millis number

              Time unit for milliseconds

            • avg_size number | string

            • avg_size_in_bytes number Required
          • shard_stats object
            Hide shard_stats attribute Show shard_stats attribute object
            • total_count number Required
        • uuid string
        • health string

          Values are green, GREEN, yellow, YELLOW, red, or RED.

        • status string

          Values are open or close.

    • _shards object Required
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number Required
        • status string
      • skipped number
    • _all object Required Additional properties
      Hide _all attributes Show _all attributes object
      • primaries object
        Hide primaries attributes Show primaries attributes object
        • completion object
          Hide completion attributes Show completion attributes object
          • size_in_bytes number Required

            Total amount, in bytes, of memory used for completion across all shards assigned to selected nodes.

          • size number | string

          • fields object
            Hide fields attribute Show fields attribute object
            • * object Additional properties
              Hide * attributes Show * attributes object
              • size
              • size_in_bytes number Required
        • docs object
          Hide docs attributes Show docs attributes object
          • count number Required

            Total number of non-deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments and may include documents from nested fields.

          • deleted number

            Total number of deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments. Elasticsearch reclaims the disk space of deleted Lucene documents when a segment is merged.

        • fielddata object
          Hide fielddata attributes Show fielddata attributes object
          • evictions number
          • memory_size number | string

          • memory_size_in_bytes number Required
          • fields object
            Hide fields attribute Show fields attribute object
            • * object Additional properties
              Hide * attributes Show * attributes object
              • memory_size
              • memory_size_in_bytes number Required
        • flush object
          Hide flush attributes Show flush attributes object
          • periodic number Required
          • total number Required
          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

        • get object
          Hide get attributes Show get attributes object
          • current number Required
          • exists_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • exists_time_in_millis number

            Time unit for milliseconds

          • exists_total number Required
          • missing_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • missing_time_in_millis number

            Time unit for milliseconds

          • missing_total number Required
          • time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • time_in_millis number

            Time unit for milliseconds

          • total number Required
        • indexing object
          Hide indexing attributes Show indexing attributes object
          • index_current number Required
          • delete_current number Required
          • delete_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • delete_time_in_millis number

            Time unit for milliseconds

          • delete_total number Required
          • is_throttled boolean Required
          • noop_update_total number Required
          • throttle_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • throttle_time_in_millis number

            Time unit for milliseconds

          • index_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • index_time_in_millis number

            Time unit for milliseconds

          • index_total number Required
          • index_failed number Required
          • types object
            Hide types attribute Show types attribute object
            • * object
          • write_load number
        • indices object Additional properties
        • merges object
          Hide merges attributes Show merges attributes object
          • current number Required
          • current_docs number Required
          • current_size string
          • current_size_in_bytes number Required
          • total number Required
          • total_auto_throttle string
          • total_auto_throttle_in_bytes number Required
          • total_docs number Required
          • total_size string
          • total_size_in_bytes number Required
          • total_stopped_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_stopped_time_in_millis number

            Time unit for milliseconds

          • total_throttled_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_throttled_time_in_millis number

            Time unit for milliseconds

          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

        • query_cache object
          Hide query_cache attributes Show query_cache attributes object
          • cache_count number Required

            Total number of entries added to the query cache across all shards assigned to selected nodes. This number includes current and evicted entries.

          • cache_size number Required

            Total number of entries currently in the query cache across all shards assigned to selected nodes.

          • evictions number Required

            Total number of query cache evictions across all shards assigned to selected nodes.

          • hit_count number Required

            Total count of query cache hits across all shards assigned to selected nodes.

          • memory_size number | string

          • memory_size_in_bytes number Required

            Total amount, in bytes, of memory used for the query cache across all shards assigned to selected nodes.

          • miss_count number Required

            Total count of query cache misses across all shards assigned to selected nodes.

          • total_count number Required

            Total count of hits and misses in the query cache across all shards assigned to selected nodes.

        • recovery object
          Hide recovery attributes Show recovery attributes object
          • current_as_source number Required
          • current_as_target number Required
          • throttle_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • throttle_time_in_millis number

            Time unit for milliseconds

        • refresh object
          Hide refresh attributes Show refresh attributes object
          • external_total number Required
          • external_total_time_in_millis number

            Time unit for milliseconds

          • listeners number Required
          • total number Required
          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

        • request_cache object
          Hide request_cache attributes Show request_cache attributes object
          • evictions number Required
          • hit_count number Required
          • memory_size string
          • memory_size_in_bytes number Required
          • miss_count number Required
        • segments object
          Hide segments attributes Show segments attributes object
          • count number Required

            Total number of segments across all shards assigned to selected nodes.

          • doc_values_memory number | string

          • doc_values_memory_in_bytes number Required

            Total amount, in bytes, of memory used for doc values across all shards assigned to selected nodes.

          • file_sizes object Required

            This object is not populated by the cluster stats API. To get information on segment files, use the node stats API.

            Hide file_sizes attribute Show file_sizes attribute object
            • * object Additional properties
              Hide * attributes Show * attributes object
              • description string Required
              • size_in_bytes number Required
              • min_size_in_bytes number
              • max_size_in_bytes number
              • average_size_in_bytes number
              • count number
          • fixed_bit_set number | string

          • fixed_bit_set_memory_in_bytes number Required

            Total amount of memory, in bytes, used by fixed bit sets across all shards assigned to selected nodes.

          • index_writer_memory number | string

          • index_writer_max_memory_in_bytes number
          • index_writer_memory_in_bytes number Required

            Total amount, in bytes, of memory used by all index writers across all shards assigned to selected nodes.

          • max_unsafe_auto_id_timestamp number Required

            Unix timestamp, in milliseconds, of the most recently retried indexing request.

          • memory number | string

          • memory_in_bytes number Required

            Total amount, in bytes, of memory used for segments across all shards assigned to selected nodes.

          • norms_memory number | string

          • norms_memory_in_bytes number Required

            Total amount, in bytes, of memory used for normalization factors across all shards assigned to selected nodes.

          • points_memory number | string

          • points_memory_in_bytes number Required

            Total amount, in bytes, of memory used for points across all shards assigned to selected nodes.

          • stored_memory number | string

          • stored_fields_memory_in_bytes number Required

            Total amount, in bytes, of memory used for stored fields across all shards assigned to selected nodes.

          • terms_memory_in_bytes number Required

            Total amount, in bytes, of memory used for terms across all shards assigned to selected nodes.

          • terms_memory number | string

          • term_vectory_memory number | string

          • term_vectors_memory_in_bytes number Required

            Total amount, in bytes, of memory used for term vectors across all shards assigned to selected nodes.

          • version_map_memory number | string

          • version_map_memory_in_bytes number Required

            Total amount, in bytes, of memory used by all version maps across all shards assigned to selected nodes.

        • store object
          Hide store attributes Show store attributes object
          • size number | string

          • size_in_bytes number Required

            Total size, in bytes, of all shards assigned to selected nodes.

          • reserved number | string

          • reserved_in_bytes number Required

            A prediction, in bytes, of how much larger the shard stores will eventually grow due to ongoing peer recoveries, restoring snapshots, and similar activities.

          • total_data_set_size number | string

          • total_data_set_size_in_bytes number

            Total data set size, in bytes, of all shards assigned to selected nodes. This includes the size of shards not stored fully on the nodes, such as the cache for partially mounted indices.

        • translog object
          Hide translog attributes Show translog attributes object
          • earliest_last_modified_age number Required
          • operations number Required
          • size string
          • size_in_bytes number Required
          • uncommitted_operations number Required
          • uncommitted_size string
          • uncommitted_size_in_bytes number Required
        • warmer object
          Hide warmer attributes Show warmer attributes object
          • current number Required
          • total number Required
          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

        • bulk object
          Hide bulk attributes Show bulk attributes object
          • total_operations number Required
          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

          • total_size number | string

          • total_size_in_bytes number Required
          • avg_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • avg_time_in_millis number

            Time unit for milliseconds

          • avg_size number | string

          • avg_size_in_bytes number Required
        • shard_stats object
          Hide shard_stats attribute Show shard_stats attribute object
          • total_count number Required
      • shards object
        Hide shards attribute Show shards attribute object
        • * array[object] Additional properties
          Hide * attributes Show * attributes object
          • commit object
            Hide commit attributes Show commit attributes object
            • generation number Required
            • id string Required
            • num_docs number Required
            • user_data object Required
          • completion object
            Hide completion attributes Show completion attributes object
            • size_in_bytes number Required

              Total amount, in bytes, of memory used for completion across all shards assigned to selected nodes.

            • size
            • fields object
          • docs object
            Hide docs attributes Show docs attributes object
            • count number Required

              Total number of non-deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments and may include documents from nested fields.

            • deleted number

              Total number of deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments. Elasticsearch reclaims the disk space of deleted Lucene documents when a segment is merged.

          • fielddata object
            Hide fielddata attributes Show fielddata attributes object
            • evictions number
            • memory_size
            • memory_size_in_bytes number Required
            • fields object
          • flush object
            Hide flush attributes Show flush attributes object
            • periodic number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis
          • get object
            Hide get attributes Show get attributes object
            • current number Required
            • exists_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • exists_time_in_millis
            • exists_total number Required
            • missing_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • missing_time_in_millis
            • missing_total number Required
            • time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • time_in_millis
            • total number Required
          • indexing object
            Hide indexing attributes Show indexing attributes object
            • index_current number Required
            • delete_current number Required
            • delete_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • delete_time_in_millis
            • delete_total number Required
            • is_throttled boolean Required
            • noop_update_total number Required
            • throttle_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • throttle_time_in_millis
            • index_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • index_time_in_millis
            • index_total number Required
            • index_failed number Required
            • types object
            • write_load number
          • mappings object
            Hide mappings attributes Show mappings attributes object
            • total_count number Required
            • total_estimated_overhead
            • total_estimated_overhead_in_bytes number Required
          • merges object
            Hide merges attributes Show merges attributes object
            • current number Required
            • current_docs number Required
            • current_size string
            • current_size_in_bytes number Required
            • total number Required
            • total_auto_throttle string
            • total_auto_throttle_in_bytes number Required
            • total_docs number Required
            • total_size string
            • total_size_in_bytes number Required
            • total_stopped_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_stopped_time_in_millis
            • total_throttled_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_throttled_time_in_millis
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis
          • shard_path object
            Hide shard_path attributes Show shard_path attributes object
            • data_path string Required
            • is_custom_data_path boolean Required
            • state_path string Required
          • query_cache object
            Hide query_cache attributes Show query_cache attributes object
            • cache_count number Required
            • cache_size number Required
            • evictions number Required
            • hit_count number Required
            • memory_size_in_bytes number Required
            • miss_count number Required
            • total_count number Required
          • recovery object
            Hide recovery attributes Show recovery attributes object
            • current_as_source number Required
            • current_as_target number Required
            • throttle_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • throttle_time_in_millis
          • refresh object
            Hide refresh attributes Show refresh attributes object
            • external_total number Required
            • external_total_time_in_millis
            • listeners number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis
          • request_cache object
            Hide request_cache attributes Show request_cache attributes object
            • evictions number Required
            • hit_count number Required
            • memory_size string
            • memory_size_in_bytes number Required
            • miss_count number Required
          • retention_leases object
            Hide retention_leases attributes Show retention_leases attributes object
            • primary_term number Required
            • version number Required
            • leases array[object] Required
          • routing object
            Hide routing attributes Show routing attributes object
            • node string Required
            • primary boolean Required
            • relocating_node
            • state string Required

              Values are UNASSIGNED, INITIALIZING, STARTED, or RELOCATING.

          • segments object
            Hide segments attributes Show segments attributes object
            • count number Required

              Total number of segments across all shards assigned to selected nodes.

            • doc_values_memory
            • doc_values_memory_in_bytes number Required

              Total amount, in bytes, of memory used for doc values across all shards assigned to selected nodes.

            • file_sizes object Required

              This object is not populated by the cluster stats API. To get information on segment files, use the node stats API.

            • fixed_bit_set
            • fixed_bit_set_memory_in_bytes number Required

              Total amount of memory, in bytes, used by fixed bit sets across all shards assigned to selected nodes.

            • index_writer_memory
            • index_writer_max_memory_in_bytes number
            • index_writer_memory_in_bytes number Required

              Total amount, in bytes, of memory used by all index writers across all shards assigned to selected nodes.

            • max_unsafe_auto_id_timestamp number Required

              Unix timestamp, in milliseconds, of the most recently retried indexing request.

            • memory
            • memory_in_bytes number Required

              Total amount, in bytes, of memory used for segments across all shards assigned to selected nodes.

            • norms_memory
            • norms_memory_in_bytes number Required

              Total amount, in bytes, of memory used for normalization factors across all shards assigned to selected nodes.

            • points_memory
            • points_memory_in_bytes number Required

              Total amount, in bytes, of memory used for points across all shards assigned to selected nodes.

            • stored_memory
            • stored_fields_memory_in_bytes number Required

              Total amount, in bytes, of memory used for stored fields across all shards assigned to selected nodes.

            • terms_memory_in_bytes number Required

              Total amount, in bytes, of memory used for terms across all shards assigned to selected nodes.

            • terms_memory
            • term_vectory_memory
            • term_vectors_memory_in_bytes number Required

              Total amount, in bytes, of memory used for term vectors across all shards assigned to selected nodes.

            • version_map_memory
            • version_map_memory_in_bytes number Required

              Total amount, in bytes, of memory used by all version maps across all shards assigned to selected nodes.

          • seq_no object
            Hide seq_no attributes Show seq_no attributes object
            • global_checkpoint number Required
            • local_checkpoint number Required
            • max_seq_no number Required
          • store object
            Hide store attributes Show store attributes object
            • size
            • size_in_bytes number Required

              Total size, in bytes, of all shards assigned to selected nodes.

            • reserved
            • reserved_in_bytes number Required

              A prediction, in bytes, of how much larger the shard stores will eventually grow due to ongoing peer recoveries, restoring snapshots, and similar activities.

            • total_data_set_size
            • total_data_set_size_in_bytes number

              Total data set size, in bytes, of all shards assigned to selected nodes. This includes the size of shards not stored fully on the nodes, such as the cache for partially mounted indices.

          • translog object
            Hide translog attributes Show translog attributes object
            • earliest_last_modified_age number Required
            • operations number Required
            • size string
            • size_in_bytes number Required
            • uncommitted_operations number Required
            • uncommitted_size string
            • uncommitted_size_in_bytes number Required
          • warmer object
            Hide warmer attributes Show warmer attributes object
            • current number Required
            • total number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis
          • bulk object
            Hide bulk attributes Show bulk attributes object
            • total_operations number Required
            • total_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total_time_in_millis
            • total_size
            • total_size_in_bytes number Required
            • avg_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • avg_time_in_millis
            • avg_size
            • avg_size_in_bytes number Required
          • shards object Generally available; Added in 7.15.0
            Hide shards attribute Show shards attribute object
            • * object Additional properties
          • shard_stats object
            Hide shard_stats attribute Show shard_stats attribute object
            • total_count number Required
          • indices object Additional properties
      • total object
        Hide total attributes Show total attributes object
        • completion object
          Hide completion attributes Show completion attributes object
          • size_in_bytes number Required

            Total amount, in bytes, of memory used for completion across all shards assigned to selected nodes.

          • size number | string

          • fields object
            Hide fields attribute Show fields attribute object
            • * object Additional properties
              Hide * attributes Show * attributes object
              • size
              • size_in_bytes number Required
        • docs object
          Hide docs attributes Show docs attributes object
          • count number Required

            Total number of non-deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments and may include documents from nested fields.

          • deleted number

            Total number of deleted documents across all primary shards assigned to selected nodes. This number is based on documents in Lucene segments. Elasticsearch reclaims the disk space of deleted Lucene documents when a segment is merged.

        • fielddata object
          Hide fielddata attributes Show fielddata attributes object
          • evictions number
          • memory_size number | string

          • memory_size_in_bytes number Required
          • fields object
            Hide fields attribute Show fields attribute object
            • * object Additional properties
              Hide * attributes Show * attributes object
              • memory_size
              • memory_size_in_bytes number Required
        • flush object
          Hide flush attributes Show flush attributes object
          • periodic number Required
          • total number Required
          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

        • get object
          Hide get attributes Show get attributes object
          • current number Required
          • exists_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • exists_time_in_millis number

            Time unit for milliseconds

          • exists_total number Required
          • missing_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • missing_time_in_millis number

            Time unit for milliseconds

          • missing_total number Required
          • time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • time_in_millis number

            Time unit for milliseconds

          • total number Required
        • indexing object
          Hide indexing attributes Show indexing attributes object
          • index_current number Required
          • delete_current number Required
          • delete_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • delete_time_in_millis number

            Time unit for milliseconds

          • delete_total number Required
          • is_throttled boolean Required
          • noop_update_total number Required
          • throttle_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • throttle_time_in_millis number

            Time unit for milliseconds

          • index_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • index_time_in_millis number

            Time unit for milliseconds

          • index_total number Required
          • index_failed number Required
          • types object
            Hide types attribute Show types attribute object
            • * object
          • write_load number
        • indices object Additional properties
        • merges object
          Hide merges attributes Show merges attributes object
          • current number Required
          • current_docs number Required
          • current_size string
          • current_size_in_bytes number Required
          • total number Required
          • total_auto_throttle string
          • total_auto_throttle_in_bytes number Required
          • total_docs number Required
          • total_size string
          • total_size_in_bytes number Required
          • total_stopped_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_stopped_time_in_millis number

            Time unit for milliseconds

          • total_throttled_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_throttled_time_in_millis number

            Time unit for milliseconds

          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

        • query_cache object
          Hide query_cache attributes Show query_cache attributes object
          • cache_count number Required

            Total number of entries added to the query cache across all shards assigned to selected nodes. This number includes current and evicted entries.

          • cache_size number Required

            Total number of entries currently in the query cache across all shards assigned to selected nodes.

          • evictions number Required

            Total number of query cache evictions across all shards assigned to selected nodes.

          • hit_count number Required

            Total count of query cache hits across all shards assigned to selected nodes.

          • memory_size number | string

          • memory_size_in_bytes number Required

            Total amount, in bytes, of memory used for the query cache across all shards assigned to selected nodes.

          • miss_count number Required

            Total count of query cache misses across all shards assigned to selected nodes.

          • total_count number Required

            Total count of hits and misses in the query cache across all shards assigned to selected nodes.

        • recovery object
          Hide recovery attributes Show recovery attributes object
          • current_as_source number Required
          • current_as_target number Required
          • throttle_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • throttle_time_in_millis number

            Time unit for milliseconds

        • refresh object
          Hide refresh attributes Show refresh attributes object
          • external_total number Required
          • external_total_time_in_millis number

            Time unit for milliseconds

          • listeners number Required
          • total number Required
          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

        • request_cache object
          Hide request_cache attributes Show request_cache attributes object
          • evictions number Required
          • hit_count number Required
          • memory_size string
          • memory_size_in_bytes number Required
          • miss_count number Required
        • segments object
          Hide segments attributes Show segments attributes object
          • count number Required

            Total number of segments across all shards assigned to selected nodes.

          • doc_values_memory number | string

          • doc_values_memory_in_bytes number Required

            Total amount, in bytes, of memory used for doc values across all shards assigned to selected nodes.

          • file_sizes object Required

            This object is not populated by the cluster stats API. To get information on segment files, use the node stats API.

            Hide file_sizes attribute Show file_sizes attribute object
            • * object Additional properties
              Hide * attributes Show * attributes object
              • description string Required
              • size_in_bytes number Required
              • min_size_in_bytes number
              • max_size_in_bytes number
              • average_size_in_bytes number
              • count number
          • fixed_bit_set number | string

          • fixed_bit_set_memory_in_bytes number Required

            Total amount of memory, in bytes, used by fixed bit sets across all shards assigned to selected nodes.

          • index_writer_memory number | string

          • index_writer_max_memory_in_bytes number
          • index_writer_memory_in_bytes number Required

            Total amount, in bytes, of memory used by all index writers across all shards assigned to selected nodes.

          • max_unsafe_auto_id_timestamp number Required

            Unix timestamp, in milliseconds, of the most recently retried indexing request.

          • memory number | string

          • memory_in_bytes number Required

            Total amount, in bytes, of memory used for segments across all shards assigned to selected nodes.

          • norms_memory number | string

          • norms_memory_in_bytes number Required

            Total amount, in bytes, of memory used for normalization factors across all shards assigned to selected nodes.

          • points_memory number | string

          • points_memory_in_bytes number Required

            Total amount, in bytes, of memory used for points across all shards assigned to selected nodes.

          • stored_memory number | string

          • stored_fields_memory_in_bytes number Required

            Total amount, in bytes, of memory used for stored fields across all shards assigned to selected nodes.

          • terms_memory_in_bytes number Required

            Total amount, in bytes, of memory used for terms across all shards assigned to selected nodes.

          • terms_memory number | string

          • term_vectory_memory number | string

          • term_vectors_memory_in_bytes number Required

            Total amount, in bytes, of memory used for term vectors across all shards assigned to selected nodes.

          • version_map_memory number | string

          • version_map_memory_in_bytes number Required

            Total amount, in bytes, of memory used by all version maps across all shards assigned to selected nodes.

        • store object
          Hide store attributes Show store attributes object
          • size number | string

          • size_in_bytes number Required

            Total size, in bytes, of all shards assigned to selected nodes.

          • reserved number | string

          • reserved_in_bytes number Required

            A prediction, in bytes, of how much larger the shard stores will eventually grow due to ongoing peer recoveries, restoring snapshots, and similar activities.

          • total_data_set_size number | string

          • total_data_set_size_in_bytes number

            Total data set size, in bytes, of all shards assigned to selected nodes. This includes the size of shards not stored fully on the nodes, such as the cache for partially mounted indices.

        • translog object
          Hide translog attributes Show translog attributes object
          • earliest_last_modified_age number Required
          • operations number Required
          • size string
          • size_in_bytes number Required
          • uncommitted_operations number Required
          • uncommitted_size string
          • uncommitted_size_in_bytes number Required
        • warmer object
          Hide warmer attributes Show warmer attributes object
          • current number Required
          • total number Required
          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

        • bulk object
          Hide bulk attributes Show bulk attributes object
          • total_operations number Required
          • total_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • total_time_in_millis number

            Time unit for milliseconds

          • total_size number | string

          • total_size_in_bytes number Required
          • avg_time string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • avg_time_in_millis number

            Time unit for milliseconds

          • avg_size number | string

          • avg_size_in_bytes number Required
        • shard_stats object
          Hide shard_stats attribute Show shard_stats attribute object
          • total_count number Required
      • uuid string
      • health string

        Values are green, GREEN, yellow, YELLOW, red, or RED.

      • status string

        Values are open or close.

GET /{index}/_stats
curl \
 --request GET 'http://api.example.com/{index}/_stats' \
 --header "Authorization: $API_KEY"
















Validate a query Generally available; Added in 1.3.0

POST /_validate/query

Validates a query without running it.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • all_shards boolean

    If true, the validation is executed on all shards instead of one random shard per index.

  • analyzer string

    Analyzer to use for the query string. This parameter can only be used when the q query string parameter is specified.

  • analyze_wildcard boolean

    If true, wildcard and prefix queries are analyzed.

  • default_operator string

    The default operator for query string query: AND or OR.

    Values are and, AND, or, or OR.

  • df string

    Field to use as default where no field prefix is given in the query string. This parameter can only be used when the q query string parameter is specified.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • explain boolean

    If true, the response returns detailed information if an error has occurred.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • lenient boolean

    If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored.

  • rewrite boolean

    If true, returns a more detailed explanation showing the actual Lucene query that will be executed.

  • q string

    Query in the Lucene query string syntax.

application/json

Body

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • explanations array[object]
      Hide explanations attributes Show explanations attributes object
      • error string
      • explanation string
      • index string Required
      • valid boolean Required
    • _shards object
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number Required
        • status string
      • skipped number
    • valid boolean Required
    • error string
POST /_validate/query
curl \
 --request POST 'http://api.example.com/_validate/query' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"query":{}}'




Validate a query Generally available; Added in 1.3.0

POST /{index}/_validate/query

Validates a query without running it.

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases to search. Supports wildcards (*). To search all data streams or indices, omit this parameter or use * or _all.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • all_shards boolean

    If true, the validation is executed on all shards instead of one random shard per index.

  • analyzer string

    Analyzer to use for the query string. This parameter can only be used when the q query string parameter is specified.

  • analyze_wildcard boolean

    If true, wildcard and prefix queries are analyzed.

  • default_operator string

    The default operator for query string query: AND or OR.

    Values are and, AND, or, or OR.

  • df string

    Field to use as default where no field prefix is given in the query string. This parameter can only be used when the q query string parameter is specified.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

    Values are all, open, closed, hidden, or none.

  • explain boolean

    If true, the response returns detailed information if an error has occurred.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • lenient boolean

    If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored.

  • rewrite boolean

    If true, returns a more detailed explanation showing the actual Lucene query that will be executed.

  • q string

    Query in the Lucene query string syntax.

application/json

Body

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • explanations array[object]
      Hide explanations attributes Show explanations attributes object
      • error string
      • explanation string
      • index string Required
      • valid boolean Required
    • _shards object
      Hide _shards attributes Show _shards attributes object
      • failed number Required
      • successful number Required
      • total number Required
      • failures array[object]
        Hide failures attributes Show failures attributes object
        • index string
        • node string
        • reason object Required

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide reason attributes Show reason attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • shard number Required
        • status string
      • skipped number
    • valid boolean Required
    • error string
POST /{index}/_validate/query
curl \
 --request POST 'http://api.example.com/{index}/_validate/query' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"query":{}}'




Create or update a lifecycle policy Generally available; Added in 6.6.0

PUT /_ilm/policy/{policy}

If the specified policy exists, it is replaced and the policy version is incremented.

NOTE: Only the latest version of the policy is stored, you cannot revert to previous versions.

Required authorization

  • Index privileges: manage
  • Cluster privileges: manage_ilm
External documentation

Path parameters

  • policy string Required

    Identifier for the policy.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

application/json

Body

  • policy object
    Hide policy attributes Show policy attributes object
    • phases object Required
      Hide phases attributes Show phases attributes object
      • cold object
        Hide cold attributes Show cold attributes object
        • actions object
          Hide actions attributes Show actions attributes object
          • allocate object
            Hide allocate attributes Show allocate attributes object
            • number_of_replicas number
            • total_shards_per_node number
            • include object
              Hide include attribute Show include attribute object
              • * string Additional properties
            • exclude object
              Hide exclude attribute Show exclude attribute object
              • * string Additional properties
            • require object
              Hide require attribute Show require attribute object
              • * string Additional properties
          • delete object
            Hide delete attribute Show delete attribute object
            • delete_searchable_snapshot boolean
          • downsample object
            Hide downsample attributes Show downsample attributes object
            • fixed_interval string Required

              A date histogram interval. Similar to Duration with additional units: w (week), M (month), q (quarter) and y (year)

            • wait_timeout string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • freeze object

            For empty Class assignments

          • forcemerge object
            Hide forcemerge attributes Show forcemerge attributes object
            • max_num_segments number Required
            • index_codec string
          • migrate object
            Hide migrate attribute Show migrate attribute object
            • enabled boolean
          • readonly object

            For empty Class assignments

          • rollover object
            Hide rollover attributes Show rollover attributes object
            • max_size number | string

            • max_primary_shard_size number | string

            • max_age string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • max_docs number
            • max_primary_shard_docs number
            • min_size number | string

            • min_primary_shard_size number | string

            • min_age string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • min_docs number
            • min_primary_shard_docs number
          • set_priority object
            Hide set_priority attribute Show set_priority attribute object
            • priority number
          • searchable_snapshot object
            Hide searchable_snapshot attributes Show searchable_snapshot attributes object
            • snapshot_repository string Required
            • force_merge_index boolean
          • shrink object
            Hide shrink attributes Show shrink attributes object
          • unfollow object

            For empty Class assignments

          • wait_for_snapshot object
            Hide wait_for_snapshot attribute Show wait_for_snapshot attribute object
            • policy string Required
        • min_age string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • delete object
        Hide delete attributes Show delete attributes object
        • actions object
          Hide actions attributes Show actions attributes object
          • allocate object
            Hide allocate attributes Show allocate attributes object
            • number_of_replicas number
            • total_shards_per_node number
            • include object
              Hide include attribute Show include attribute object
              • * string Additional properties
            • exclude object
              Hide exclude attribute Show exclude attribute object
              • * string Additional properties
            • require object
              Hide require attribute Show require attribute object
              • * string Additional properties
          • delete object
            Hide delete attribute Show delete attribute object
            • delete_searchable_snapshot boolean
          • downsample object
            Hide downsample attributes Show downsample attributes object
            • fixed_interval string Required

              A date histogram interval. Similar to Duration with additional units: w (week), M (month), q (quarter) and y (year)

            • wait_timeout string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • freeze object

            For empty Class assignments

          • forcemerge object
            Hide forcemerge attributes Show forcemerge attributes object
            • max_num_segments number Required
            • index_codec string
          • migrate object
            Hide migrate attribute Show migrate attribute object
            • enabled boolean
          • readonly object

            For empty Class assignments

          • rollover object
            Hide rollover attributes Show rollover attributes object
            • max_size number | string

            • max_primary_shard_size number | string

            • max_age string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • max_docs number
            • max_primary_shard_docs number
            • min_size number | string

            • min_primary_shard_size number | string

            • min_age string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • min_docs number
            • min_primary_shard_docs number
          • set_priority object
            Hide set_priority attribute Show set_priority attribute object
            • priority number
          • searchable_snapshot object
            Hide searchable_snapshot attributes Show searchable_snapshot attributes object
            • snapshot_repository string Required
            • force_merge_index boolean
          • shrink object
            Hide shrink attributes Show shrink attributes object
          • unfollow object

            For empty Class assignments

          • wait_for_snapshot object
            Hide wait_for_snapshot attribute Show wait_for_snapshot attribute object
            • policy string Required
        • min_age string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • frozen object
        Hide frozen attributes Show frozen attributes object
        • actions object
          Hide actions attributes Show actions attributes object
          • allocate object
            Hide allocate attributes Show allocate attributes object
            • number_of_replicas number
            • total_shards_per_node number
            • include object
              Hide include attribute Show include attribute object
              • * string Additional properties
            • exclude object
              Hide exclude attribute Show exclude attribute object
              • * string Additional properties
            • require object
              Hide require attribute Show require attribute object
              • * string Additional properties
          • delete object
            Hide delete attribute Show delete attribute object
            • delete_searchable_snapshot boolean
          • downsample object
            Hide downsample attributes Show downsample attributes object
            • fixed_interval string Required

              A date histogram interval. Similar to Duration with additional units: w (week), M (month), q (quarter) and y (year)

            • wait_timeout string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • freeze object

            For empty Class assignments

          • forcemerge object
            Hide forcemerge attributes Show forcemerge attributes object
            • max_num_segments number Required
            • index_codec string
          • migrate object
            Hide migrate attribute Show migrate attribute object
            • enabled boolean
          • readonly object

            For empty Class assignments

          • rollover object
            Hide rollover attributes Show rollover attributes object
            • max_size number | string

            • max_primary_shard_size number | string

            • max_age string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • max_docs number
            • max_primary_shard_docs number
            • min_size number | string

            • min_primary_shard_size number | string

            • min_age string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • min_docs number
            • min_primary_shard_docs number
          • set_priority object
            Hide set_priority attribute Show set_priority attribute object
            • priority number
          • searchable_snapshot object
            Hide searchable_snapshot attributes Show searchable_snapshot attributes object
            • snapshot_repository string Required
            • force_merge_index boolean
          • shrink object
            Hide shrink attributes Show shrink attributes object
          • unfollow object

            For empty Class assignments

          • wait_for_snapshot object
            Hide wait_for_snapshot attribute Show wait_for_snapshot attribute object
            • policy string Required
        • min_age string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • hot object
        Hide hot attributes Show hot attributes object
        • actions object
          Hide actions attributes Show actions attributes object
          • allocate object
            Hide allocate attributes Show allocate attributes object
            • number_of_replicas number
            • total_shards_per_node number
            • include object
              Hide include attribute Show include attribute object
              • * string Additional properties
            • exclude object
              Hide exclude attribute Show exclude attribute object
              • * string Additional properties
            • require object
              Hide require attribute Show require attribute object
              • * string Additional properties
          • delete object
            Hide delete attribute Show delete attribute object
            • delete_searchable_snapshot boolean
          • downsample object
            Hide downsample attributes Show downsample attributes object
            • fixed_interval string Required

              A date histogram interval. Similar to Duration with additional units: w (week), M (month), q (quarter) and y (year)

            • wait_timeout string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • freeze object

            For empty Class assignments

          • forcemerge object
            Hide forcemerge attributes Show forcemerge attributes object
            • max_num_segments number Required
            • index_codec string
          • migrate object
            Hide migrate attribute Show migrate attribute object
            • enabled boolean
          • readonly object

            For empty Class assignments

          • rollover object
            Hide rollover attributes Show rollover attributes object
            • max_size number | string

            • max_primary_shard_size number | string

            • max_age string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • max_docs number
            • max_primary_shard_docs number
            • min_size number | string

            • min_primary_shard_size number | string

            • min_age string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • min_docs number
            • min_primary_shard_docs number
          • set_priority object
            Hide set_priority attribute Show set_priority attribute object
            • priority number
          • searchable_snapshot object
            Hide searchable_snapshot attributes Show searchable_snapshot attributes object
            • snapshot_repository string Required
            • force_merge_index boolean
          • shrink object
            Hide shrink attributes Show shrink attributes object
          • unfollow object

            For empty Class assignments

          • wait_for_snapshot object
            Hide wait_for_snapshot attribute Show wait_for_snapshot attribute object
            • policy string Required
        • min_age string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • warm object
        Hide warm attributes Show warm attributes object
        • actions object
          Hide actions attributes Show actions attributes object
          • allocate object
            Hide allocate attributes Show allocate attributes object
            • number_of_replicas number
            • total_shards_per_node number
            • include object
              Hide include attribute Show include attribute object
              • * string Additional properties
            • exclude object
              Hide exclude attribute Show exclude attribute object
              • * string Additional properties
            • require object
              Hide require attribute Show require attribute object
              • * string Additional properties
          • delete object
            Hide delete attribute Show delete attribute object
            • delete_searchable_snapshot boolean
          • downsample object
            Hide downsample attributes Show downsample attributes object
            • fixed_interval string Required

              A date histogram interval. Similar to Duration with additional units: w (week), M (month), q (quarter) and y (year)

            • wait_timeout string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • freeze object

            For empty Class assignments

          • forcemerge object
            Hide forcemerge attributes Show forcemerge attributes object
            • max_num_segments number Required
            • index_codec string
          • migrate object
            Hide migrate attribute Show migrate attribute object
            • enabled boolean
          • readonly object

            For empty Class assignments

          • rollover object
            Hide rollover attributes Show rollover attributes object
            • max_size number | string

            • max_primary_shard_size number | string

            • max_age string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • max_docs number
            • max_primary_shard_docs number
            • min_size number | string

            • min_primary_shard_size number | string

            • min_age string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • min_docs number
            • min_primary_shard_docs number
          • set_priority object
            Hide set_priority attribute Show set_priority attribute object
            • priority number
          • searchable_snapshot object
            Hide searchable_snapshot attributes Show searchable_snapshot attributes object
            • snapshot_repository string Required
            • force_merge_index boolean
          • shrink object
            Hide shrink attributes Show shrink attributes object
          • unfollow object

            For empty Class assignments

          • wait_for_snapshot object
            Hide wait_for_snapshot attribute Show wait_for_snapshot attribute object
            • policy string Required
        • min_age string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_ilm/policy/{policy}
curl \
 --request PUT 'http://api.example.com/_ilm/policy/{policy}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"policy\": {\n    \"_meta\": {\n      \"description\": \"used for nginx log\",\n      \"project\": {\n        \"name\": \"myProject\",\n        \"department\": \"myDepartment\"\n      }\n    },\n    \"phases\": {\n      \"warm\": {\n        \"min_age\": \"10d\",\n        \"actions\": {\n          \"forcemerge\": {\n            \"max_num_segments\": 1\n          }\n        }\n      },\n      \"delete\": {\n        \"min_age\": \"30d\",\n        \"actions\": {\n          \"delete\": {}\n        }\n      }\n    }\n  }\n}"'
Request example
Run `PUT _ilm/policy/my_policy` to create a new policy with arbitrary metadata.
{
  "policy": {
    "_meta": {
      "description": "used for nginx log",
      "project": {
        "name": "myProject",
        "department": "myDepartment"
      }
    },
    "phases": {
      "warm": {
        "min_age": "10d",
        "actions": {
          "forcemerge": {
            "max_num_segments": 1
          }
        }
      },
      "delete": {
        "min_age": "30d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}
Response examples (200)
A successful response when creating a new lifecycle policy.
{
  "acknowledged": true
}




Explain the lifecycle state Generally available; Added in 6.6.0

GET /{index}/_ilm/explain

Get the current lifecycle status for one or more indices. For data streams, the API retrieves the current lifecycle status for the stream's backing indices.

The response indicates when the index entered each lifecycle state, provides the definition of the running phase, and information about any failures.

Required authorization

  • Index privileges: view_index_metadata,manage_ilm

Path parameters

  • index string Required

    Comma-separated list of data streams, indices, and aliases to target. Supports wildcards (*). To target all data streams and indices, use * or _all.

Query parameters

  • only_errors boolean

    Filters the returned indices to only indices that are managed by ILM and are in an error state, either due to an encountering an error while executing the policy, or attempting to use a policy that does not exist.

  • only_managed boolean

    Filters the returned indices to only indices that are managed by ILM.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • indices object Required
GET /{index}/_ilm/explain
curl \
 --request GET 'http://api.example.com/{index}/_ilm/explain' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response when retrieving the current ILM status for an index.
{
  "indices": {
    "my-index-000001": {
      "index": "my-index-000001",
      "index_creation_date_millis": 1538475653281,
      "index_creation_date": "2018-10-15T13:45:21.981Z",
      "time_since_index_creation": "15s",
      "managed": true,
      "policy": "my_policy",
      "lifecycle_date_millis": 1538475653281,
      "lifecycle_date": "2018-10-15T13:45:21.981Z",
      "age": "15s",
      "phase": "new",
      "phase_time_millis": 1538475653317,
      "phase_time": "2018-10-15T13:45:22.577Z",
      "action": "complete"
      "action_time_millis": 1538475653317,
      "action_time": "2018-10-15T13:45:22.577Z",
      "step": "complete",
      "step_time_millis": 1538475653317,
      "step_time": "2018-10-15T13:45:22.577Z"
    }
  }
}




Get the ILM status Generally available; Added in 6.6.0

GET /_ilm/status

Get the current index lifecycle management status.

Required authorization

  • Cluster privileges: read_ilm

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • operation_mode string Required

      Values are RUNNING, STOPPING, or STOPPED.

GET /_ilm/status
curl \
 --request GET 'http://api.example.com/_ilm/status' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response when retrieving the current ILM status.
{
  "operation_mode": "RUNNING"
}




Move to a lifecycle step Generally available; Added in 6.6.0

POST /_ilm/move/{index}

Manually move an index into a specific step in the lifecycle policy and run that step.

WARNING: This operation can result in the loss of data. Manually moving an index into a specific step runs that step even if it has already been performed. This is a potentially destructive action and this should be considered an expert level API.

You must specify both the current step and the step to be executed in the body of the request. The request will fail if the current step does not match the step currently running for the index This is to prevent the index from being moved from an unexpected step into the next step.

When specifying the target (next_step) to which the index will be moved, either the name or both the action and name fields are optional. If only the phase is specified, the index will move to the first step of the first action in the target phase. If the phase and action are specified, the index will move to the first step of the specified action in the specified phase. Only actions specified in the ILM policy are considered valid. An index cannot move to a step that is not part of its policy.

Required authorization

  • Index privileges: manage_ilm

Path parameters

  • index string Required

    The name of the index whose lifecycle step is to change

application/json

Body

  • current_step object Required
    Hide current_step attributes Show current_step attributes object
    • action string

      The optional action to which the index will be moved.

    • name string

      The optional step name to which the index will be moved.

    • phase string Required
  • next_step object Required
    Hide next_step attributes Show next_step attributes object
    • action string

      The optional action to which the index will be moved.

    • name string

      The optional step name to which the index will be moved.

    • phase string Required

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /_ilm/move/{index}
curl \
 --request POST 'http://api.example.com/_ilm/move/{index}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"current_step\": {\n    \"phase\": \"new\",\n    \"action\": \"complete\",\n    \"name\": \"complete\"\n  },\n  \"next_step\": {\n    \"phase\": \"warm\",\n    \"action\": \"forcemerge\",\n    \"name\": \"forcemerge\"\n  }\n}"'
Request examples
Run `POST _ilm/move/my-index-000001` to move `my-index-000001` from the initial step to the `forcemerge` step.
{
  "current_step": {
    "phase": "new",
    "action": "complete",
    "name": "complete"
  },
  "next_step": {
    "phase": "warm",
    "action": "forcemerge",
    "name": "forcemerge"
  }
}
Run `POST _ilm/move/my-index-000001` to move `my-index-000001` from the end of hot phase into the start of warm.
{
  "current_step": {
    "phase": "hot",
    "action": "complete",
    "name": "complete"
  },
  "next_step": {
    "phase": "warm"
  }
}
Response examples (200)
A successful response when running a specific step in a lifecycle policy.
{
  "acknowledged": true
}




Retry a policy Generally available; Added in 6.6.0

POST /{index}/_ilm/retry

Retry running the lifecycle policy for an index that is in the ERROR step. The API sets the policy back to the step where the error occurred and runs the step. Use the explain lifecycle state API to determine whether an index is in the ERROR step.

Required authorization

  • Index privileges: manage_ilm

Path parameters

  • index string Required

    The name of the indices (comma-separated) whose failed lifecycle step is to be retry

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /{index}/_ilm/retry
curl \
 --request POST 'http://api.example.com/{index}/_ilm/retry' \
 --header "Authorization: $API_KEY"








Inference

Inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.

Perform chat completion inference Generally available; Added in 8.18.0

POST /_inference/chat_completion/{inference_id}/_stream

The chat completion inference API enables real-time responses for chat completion tasks by delivering answers incrementally, reducing response times during computation. It only works with the chat_completion task type for openai, elastic and googlevertexai inference services.

NOTE: The chat_completion task type is only available within the _stream API and only supports streaming. The Chat completion inference API and the Stream inference API differ in their response structure and capabilities. The Chat completion inference API provides more comprehensive customization options through more fields and function calling support. If you use the openai service or the elastic service, use the Chat completion inference API.

Path parameters

  • inference_id string Required

    The inference Id

Query parameters

  • timeout string

    Specifies the amount of time to wait for the inference request to complete.

    Values are -1 or 0.

application/json

Body Required

  • messages array[object] Required

    A list of objects representing the conversation. Requests should generally only add new messages from the user (role user). The other message roles (assistant, system, or tool) should generally only be copied from the response to a previous completion request, such that the messages array is built up throughout a conversation.

    An object representing part of the conversation.

    Hide messages attributes Show messages attributes object
    • content string | array[object]

      One of:
    • role string Required

      The role of the message author. Valid values are user, assistant, system, and tool.

    • tool_call_id string
    • tool_calls array[object]

      Only for assistant role messages. The tool calls generated by the model. If it's specified, the content field is optional. Example:

      {
        "tool_calls": [
            {
                "id": "call_KcAjWtAww20AihPHphUh46Gd",
                "type": "function",
                "function": {
                    "name": "get_current_weather",
                    "arguments": "{\"location\":\"Boston, MA\"}"
                }
            }
        ]
      }
      

      A tool call generated by the model.

      Hide tool_calls attributes Show tool_calls attributes object
      • id string Required
      • function object Required

        The function that the model called.

        Hide function attributes Show function attributes object
        • arguments string Required

          The arguments to call the function with in JSON format.

        • name string Required

          The name of the function to call.

      • type string Required

        The type of the tool call.

  • model string

    The ID of the model to use.

  • max_completion_tokens number

    The upper bound limit for the number of tokens that can be generated for a completion request.

  • stop array[string]

    A sequence of strings to control when the model should stop generating additional tokens.

  • temperature number

    The sampling temperature to use.

  • tool_choice string | object

    One of:
  • tools array[object]

    A list of tools that the model can call. Example:

    {
      "tools": [
          {
              "type": "function",
              "function": {
                  "name": "get_price_of_item",
                  "description": "Get the current price of an item",
                  "parameters": {
                      "type": "object",
                      "properties": {
                          "item": {
                              "id": "12345"
                          },
                          "unit": {
                              "type": "currency"
                          }
                      }
                  }
              }
          }
      ]
    }
    

    A list of tools that the model can call.

    Hide tools attributes Show tools attributes object
    • type string Required

      The type of tool.

    • function object Required

      The completion tool function definition.

      Hide function attributes Show function attributes object
      • description string

        A description of what the function does. This is used by the model to choose when and how to call the function.

      • name string Required

        The name of the function.

      • parameters object

        The parameters the functional accepts. This should be formatted as a JSON object.

      • strict boolean

        Whether to enable schema adherence when generating the function call.

  • top_p number

    Nucleus sampling, an alternative to sampling with temperature.

Responses

  • 200 application/json
POST /_inference/chat_completion/{inference_id}/_stream
curl \
 --request POST 'http://api.example.com/_inference/chat_completion/{inference_id}/_stream' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"model\": \"gpt-4o\",\n  \"messages\": [\n      {\n          \"role\": \"user\",\n          \"content\": \"What is Elastic?\"\n      }\n  ]\n}"'
Run `POST _inference/chat_completion/openai-completion/_stream` to perform a chat completion on the example question with streaming.
{
  "model": "gpt-4o",
  "messages": [
      {
          "role": "user",
          "content": "What is Elastic?"
      }
  ]
}
Run `POST _inference/chat_completion/openai-completion/_stream` to perform a chat completion using an Assistant message with `tool_calls`.
{
  "messages": [
      {
          "role": "assistant",
          "content": "Let's find out what the weather is",
          "tool_calls": [ 
              {
                  "id": "call_KcAjWtAww20AihPHphUh46Gd",
                  "type": "function",
                  "function": {
                      "name": "get_current_weather",
                      "arguments": "{\"location\":\"Boston, MA\"}"
                  }
              }
          ]
      },
      { 
          "role": "tool",
          "content": "The weather is cold",
          "tool_call_id": "call_KcAjWtAww20AihPHphUh46Gd"
      }
  ]
}
Run `POST _inference/chat_completion/openai-completion/_stream` to perform a chat completion using a User message with `tools` and `tool_choice`.
{
  "messages": [
      {
          "role": "user",
          "content": [
              {
                  "type": "text",
                  "text": "What's the price of a scarf?"
              }
          ]
      }
  ],
  "tools": [
      {
          "type": "function",
          "function": {
              "name": "get_current_price",
              "description": "Get the current price of a item",
              "parameters": {
                  "type": "object",
                  "properties": {
                      "item": {
                          "id": "123"
                      }
                  }
              }
          }
      }
  ],
  "tool_choice": {
      "type": "function",
      "function": {
          "name": "get_current_price"
      }
  }
}
Response examples (200)
A successful response when performing a chat completion task using a User message with `tools` and `tool_choice`.
event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":"","role":"assistant"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}

event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":Elastic"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}

event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":" is"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}

(...)

event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk","usage":{"completion_tokens":28,"prompt_tokens":16,"total_tokens":44}}} 

event: message
data: [DONE]

Perform completion inference on the service Generally available; Added in 8.11.0

POST /_inference/completion/{inference_id}

Path parameters

  • inference_id string Required

    The inference Id

Query parameters

  • timeout string

    Specifies the amount of time to wait for the inference request to complete.

    Values are -1 or 0.

application/json

Body

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • completion array[object] Required

      The completion result object

      Hide completion attribute Show completion attribute object
      • result string Required
POST /_inference/completion/{inference_id}
curl \
 --request POST 'http://api.example.com/_inference/completion/{inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"input\": \"What is Elastic?\"\n}"'
Request example
Run `POST _inference/completion/openai_chat_completions` to perform a completion on the example question.
{
  "input": "What is Elastic?"
}
Response examples (200)
A successful response from `POST _inference/completion/openai_chat_completions`.
{
  "completion": [
    {
      "result": "Elastic is a company that provides a range of software solutions for search, logging, security, and analytics. Their flagship product is Elasticsearch, an open-source, distributed search engine that allows users to search, analyze, and visualize large volumes of data in real-time. Elastic also offers products such as Kibana, a data visualization tool, and Logstash, a log management and pipeline tool, as well as various other tools and solutions for data analysis and management."
    }
  ]
}




Create an inference endpoint Generally available; Added in 8.11.0

PUT /_inference/{inference_id}

IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Mistral, Azure OpenAI, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.

The following integrations are available through the inference API. You can find the available task types next to the integration name:

  • AlibabaCloud AI Search (completion, rerank, sparse_embedding, text_embedding)
  • Amazon Bedrock (completion, text_embedding)
  • Anthropic (completion)
  • Azure AI Studio (completion, text_embedding)
  • Azure OpenAI (completion, text_embedding)
  • Cohere (completion, rerank, text_embedding)
  • Elasticsearch (rerank, sparse_embedding, text_embedding - this service is for built-in models and models uploaded through Eland)
  • ELSER (sparse_embedding)
  • Google AI Studio (completion, text_embedding)
  • Google Vertex AI (rerank, text_embedding)
  • Hugging Face (text_embedding)
  • Mistral (text_embedding)
  • OpenAI (chat_completion, completion, text_embedding)
  • VoyageAI (text_embedding, rerank)
  • Watsonx inference integration (text_embedding)
  • JinaAI (text_embedding, rerank)

Required authorization

  • Cluster privileges: manage_inference

Path parameters

  • inference_id string Required

    The inference Id

application/json

Body Required

  • chunking_settings object

    Chunking configuration object

    Hide chunking_settings attributes Show chunking_settings attributes object
    • max_chunk_size number

      The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

    • overlap number

      The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

    • sentence_overlap number

      The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

    • strategy string

      The chunking strategy: sentence or word.

  • service string Required

    The service type

  • service_settings object Required
  • task_settings object

Responses

  • 200 application/json
    Hide response attributes Show response attributes object

    Represents an inference endpoint as returned by the GET API

    • chunking_settings object

      Chunking configuration object

      Hide chunking_settings attributes Show chunking_settings attributes object
      • max_chunk_size number

        The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

      • overlap number

        The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

      • sentence_overlap number

        The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

      • strategy string

        The chunking strategy: sentence or word.

    • service string Required

      The service type

    • service_settings object Required
    • task_settings object
    • inference_id string Required

      The inference Id

    • task_type string Required

      Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.

PUT /_inference/{inference_id}
curl \
 --request PUT 'http://api.example.com/_inference/{inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n \"service\": \"cohere\",\n \"service_settings\": {\n   \"model_id\": \"rerank-english-v3.0\",\n   \"api_key\": \"{{COHERE_API_KEY}}\"\n }\n}"'
Request example
An example body for a `PUT _inference/rerank/my-rerank-model` request.
{
 "service": "cohere",
 "service_settings": {
   "model_id": "rerank-english-v3.0",
   "api_key": "{{COHERE_API_KEY}}"
 }
}

Perform inference on the service Generally available; Added in 8.11.0

POST /_inference/{inference_id}

This API enables you to use machine learning models to perform specific tasks on data that you provide as an input. It returns a response with the results of the tasks. The inference endpoint you use can perform one specific task that has been defined when the endpoint was created with the create inference API.

For details about using this API with a service, such as Amazon Bedrock, Anthropic, or HuggingFace, refer to the service-specific documentation.


The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.

Required authorization

  • Cluster privileges: monitor_inference

Path parameters

  • inference_id string Required

    The unique identifier for the inference endpoint.

Query parameters

  • timeout string

    The amount of time to wait for the inference request to complete.

    Values are -1 or 0.

application/json

Body

  • query string

    The query input, which is required only for the rerank task. It is not required for other tasks.

  • input string | array[string] Required

    The text on which you want to perform the inference task. It can be a single string or an array.


    Inference endpoints for the completion task type currently only support a single string as input.

  • task_settings object

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • text_embedding_bytes array[object]

      The text embedding result object for byte representation

      Hide text_embedding_bytes attribute Show text_embedding_bytes attribute object
      • embedding array[number] Required

        Text Embedding results containing bytes are represented as Dense Vectors of bytes.

    • text_embedding_bits array[object]

      The text embedding result object for byte representation

      Hide text_embedding_bits attribute Show text_embedding_bits attribute object
      • embedding array[number] Required

        Text Embedding results containing bytes are represented as Dense Vectors of bytes.

    • text_embedding array[object]

      The text embedding result object

      Hide text_embedding attribute Show text_embedding attribute object
      • embedding array[number] Required

        Text Embedding results are represented as Dense Vectors of floats.

    • sparse_embedding array[object]
      Hide sparse_embedding attribute Show sparse_embedding attribute object
      • embedding object Required

        Sparse Embedding tokens are represented as a dictionary of string to double.

        Hide embedding attribute Show embedding attribute object
        • * number Additional properties
    • completion array[object]

      The completion result object

      Hide completion attribute Show completion attribute object
      • result string Required
    • rerank array[object]

      The rerank result object representing a single ranked document id: the original index of the document in the request relevance_score: the relevance_score of the document relative to the query text: Optional, the text of the document, if requested

      Hide rerank attributes Show rerank attributes object
      • index number Required
      • relevance_score number Required
      • text string
POST /_inference/{inference_id}
curl \
 --request POST 'http://api.example.com/_inference/{inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"query":"string","input":"string","task_settings":{}}'




Get an inference endpoint Generally available; Added in 8.11.0

GET /_inference/{task_type}/{inference_id}

Path parameters

  • task_type string Required

    The task type

    Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.

  • inference_id string Required

    The inference Id

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • endpoints array[object] Required

      Represents an inference endpoint as returned by the GET API

      Hide endpoints attributes Show endpoints attributes object

      Represents an inference endpoint as returned by the GET API

      • chunking_settings object

        Chunking configuration object

        Hide chunking_settings attributes Show chunking_settings attributes object
        • max_chunk_size number

          The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

        • overlap number

          The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

        • sentence_overlap number

          The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

        • strategy string

          The chunking strategy: sentence or word.

      • service string Required

        The service type

      • service_settings object Required
      • task_settings object
      • inference_id string Required

        The inference Id

      • task_type string Required

        Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.

GET /_inference/{task_type}/{inference_id}
curl \
 --request GET 'http://api.example.com/_inference/{task_type}/{inference_id}' \
 --header "Authorization: $API_KEY"

Create an inference endpoint Generally available; Added in 8.11.0

PUT /_inference/{task_type}/{inference_id}

IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Mistral, Azure OpenAI, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.

The following integrations are available through the inference API. You can find the available task types next to the integration name:

  • AlibabaCloud AI Search (completion, rerank, sparse_embedding, text_embedding)
  • Amazon Bedrock (completion, text_embedding)
  • Anthropic (completion)
  • Azure AI Studio (completion, text_embedding)
  • Azure OpenAI (completion, text_embedding)
  • Cohere (completion, rerank, text_embedding)
  • Elasticsearch (rerank, sparse_embedding, text_embedding - this service is for built-in models and models uploaded through Eland)
  • ELSER (sparse_embedding)
  • Google AI Studio (completion, text_embedding)
  • Google Vertex AI (rerank, text_embedding)
  • Hugging Face (text_embedding)
  • Mistral (text_embedding)
  • OpenAI (chat_completion, completion, text_embedding)
  • VoyageAI (text_embedding, rerank)
  • Watsonx inference integration (text_embedding)
  • JinaAI (text_embedding, rerank)

Required authorization

  • Cluster privileges: manage_inference

Path parameters

  • task_type string Required

    The task type. Refer to the integration list in the API description for the available task types.

    Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.

  • inference_id string Required

    The inference Id

application/json

Body Required

  • chunking_settings object

    Chunking configuration object

    Hide chunking_settings attributes Show chunking_settings attributes object
    • max_chunk_size number

      The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

    • overlap number

      The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

    • sentence_overlap number

      The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

    • strategy string

      The chunking strategy: sentence or word.

  • service string Required

    The service type

  • service_settings object Required
  • task_settings object

Responses

  • 200 application/json
    Hide response attributes Show response attributes object

    Represents an inference endpoint as returned by the GET API

    • chunking_settings object

      Chunking configuration object

      Hide chunking_settings attributes Show chunking_settings attributes object
      • max_chunk_size number

        The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

      • overlap number

        The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

      • sentence_overlap number

        The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

      • strategy string

        The chunking strategy: sentence or word.

    • service string Required

      The service type

    • service_settings object Required
    • task_settings object
    • inference_id string Required

      The inference Id

    • task_type string Required

      Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.

PUT /_inference/{task_type}/{inference_id}
curl \
 --request PUT 'http://api.example.com/_inference/{task_type}/{inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n \"service\": \"cohere\",\n \"service_settings\": {\n   \"model_id\": \"rerank-english-v3.0\",\n   \"api_key\": \"{{COHERE_API_KEY}}\"\n }\n}"'
Request example
An example body for a `PUT _inference/rerank/my-rerank-model` request.
{
 "service": "cohere",
 "service_settings": {
   "model_id": "rerank-english-v3.0",
   "api_key": "{{COHERE_API_KEY}}"
 }
}




































Create an Elasticsearch inference endpoint Generally available; Added in 8.13.0

PUT /_inference/{task_type}/{elasticsearch_inference_id}

Create an inference endpoint to perform an inference task with the elasticsearch service.


Your Elasticsearch deployment contains preconfigured ELSER and E5 inference endpoints, you only need to create the enpoints using the API if you want to customize the settings.

If you use the ELSER or the E5 model through the elasticsearch service, the API request will automatically download and deploy the model if it isn't downloaded yet.


You might see a 502 bad gateway error in the response when using the Kibana Console. This error usually just reflects a timeout, while the model downloads in the background. You can check the download progress in the Machine Learning UI. If using the Python client, you can set the timeout parameter to a higher value.

After creating the endpoint, wait for the model deployment to complete before using it. To verify the deployment status, use the get trained model statistics API. Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count". Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.

Required authorization

  • Cluster privileges: manage_inference

Path parameters

  • task_type string Required

    The type of the inference task that the model will perform.

    Values are rerank, sparse_embedding, or text_embedding.

  • elasticsearch_inference_id string Required

    The unique identifier of the inference endpoint. The must not match the model_id.

application/json

Body

  • chunking_settings object

    Chunking configuration object

    Hide chunking_settings attributes Show chunking_settings attributes object
    • max_chunk_size number

      The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

    • overlap number

      The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

    • sentence_overlap number

      The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

    • strategy string

      The chunking strategy: sentence or word.

  • service string Required

    Value is elasticsearch.

  • service_settings object Required
    Hide service_settings attributes Show service_settings attributes object
    • adaptive_allocations object
      Hide adaptive_allocations attributes Show adaptive_allocations attributes object
      • enabled boolean

        Turn on adaptive_allocations.

      • max_number_of_allocations number

        The maximum number of allocations to scale to. If set, it must be greater than or equal to min_number_of_allocations.

      • min_number_of_allocations number

        The minimum number of allocations to scale to. If set, it must be greater than or equal to 0. If not defined, the deployment scales to 0.

    • deployment_id string

      The deployment identifier for a trained model deployment. When deployment_id is used the model_id is optional.

    • model_id string Required

      The name of the model to use for the inference task. It can be the ID of a built-in model (for example, .multilingual-e5-small for E5) or a text embedding model that was uploaded by using the Eland client.

      External documentation
    • num_allocations number

      The total number of allocations that are assigned to the model across machine learning nodes. Increasing this value generally increases the throughput. If adaptive allocations are enabled, do not set this value because it's automatically set.

    • num_threads number Required

      The number of threads used by each model allocation during inference. This setting generally increases the speed per inference request. The inference process is a compute-bound process; threads_per_allocations must not exceed the number of available allocated processors per node. The value must be a power of 2. The maximum value is 32.

  • task_settings object
    Hide task_settings attribute Show task_settings attribute object
    • return_documents boolean

      For a rerank task, return the document instead of only the index.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • chunking_settings object

      Chunking configuration object

      Hide chunking_settings attributes Show chunking_settings attributes object
      • max_chunk_size number

        The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

      • overlap number

        The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

      • sentence_overlap number

        The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

      • strategy string

        The chunking strategy: sentence or word.

    • service string Required

      The service type

    • service_settings object Required
    • task_settings object
    • inference_id string Required

      The inference Id

    • task_type string Required

      Values are sparse_embedding, text_embedding, or rerank.

PUT /_inference/{task_type}/{elasticsearch_inference_id}
curl \
 --request PUT 'http://api.example.com/_inference/{task_type}/{elasticsearch_inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"service\": \"elasticsearch\",\n    \"service_settings\": {\n        \"adaptive_allocations\": { \n        \"enabled\": true,\n        \"min_number_of_allocations\": 1,\n        \"max_number_of_allocations\": 4\n        },\n        \"num_threads\": 1,\n        \"model_id\": \".elser_model_2\" \n    }\n}"'
Run `PUT _inference/sparse_embedding/my-elser-model` to create an inference endpoint that performs a `sparse_embedding` task. The `model_id` must be the ID of one of the built-in ELSER models. The API will automatically download the ELSER model if it isn't already downloaded and then deploy the model.
{
    "service": "elasticsearch",
    "service_settings": {
        "adaptive_allocations": { 
        "enabled": true,
        "min_number_of_allocations": 1,
        "max_number_of_allocations": 4
        },
        "num_threads": 1,
        "model_id": ".elser_model_2" 
    }
}
Run `PUT _inference/rerank/my-elastic-rerank` to create an inference endpoint that performs a rerank task using the built-in Elastic Rerank cross-encoder model. The `model_id` must be `.rerank-v1`, which is the ID of the built-in Elastic Rerank model. The API will automatically download the Elastic Rerank model if it isn't already downloaded and then deploy the model. Once deployed, the model can be used for semantic re-ranking with a `text_similarity_reranker` retriever.
{
    "service": "elasticsearch",
    "service_settings": {
        "model_id": ".rerank-v1", 
        "num_threads": 1,
        "adaptive_allocations": { 
        "enabled": true,
        "min_number_of_allocations": 1,
        "max_number_of_allocations": 4
        }
    }
}
Run `PUT _inference/text_embedding/my-e5-model` to create an inference endpoint that performs a `text_embedding` task. The `model_id` must be the ID of one of the built-in E5 models. The API will automatically download the E5 model if it isn't already downloaded and then deploy the model.
{
    "service": "elasticsearch",
    "service_settings": {
        "num_allocations": 1,
        "num_threads": 1,
        "model_id": ".multilingual-e5-small" 
    }
}
Run `PUT _inference/text_embedding/my-msmarco-minilm-model` to create an inference endpoint that performs a `text_embedding` task with a model that was uploaded by Eland.
{
    "service": "elasticsearch",
    "service_settings": {
        "num_allocations": 1,
        "num_threads": 1,
        "model_id": "msmarco-MiniLM-L12-cos-v5" 
    }
}
Run `PUT _inference/text_embedding/my-e5-model` to create an inference endpoint that performs a `text_embedding` task and to configure adaptive allocations. The API request will automatically download the E5 model if it isn't already downloaded and then deploy the model.
{
    "service": "elasticsearch",
    "service_settings": {
        "adaptive_allocations": {
        "enabled": true,
        "min_number_of_allocations": 3,
        "max_number_of_allocations": 10
        },
        "num_threads": 1,
        "model_id": ".multilingual-e5-small"
    }
}
Run `PUT _inference/sparse_embedding/use_existing_deployment` to use an already existing model deployment when creating an inference endpoint.
{
    "service": "elasticsearch",
    "service_settings": {
        "deployment_id": ".elser_model_2"
    }
}
Response examples (200)
A successful response from `PUT _inference/sparse_embedding/use_existing_deployment`. It contains the model ID and the threads and allocations settings from the model deployment.
{
  "inference_id": "use_existing_deployment",
  "task_type": "sparse_embedding",
  "service": "elasticsearch",
  "service_settings": {
    "num_allocations": 2,
    "num_threads": 1,
    "model_id": ".elser_model_2",
    "deployment_id": ".elser_model_2"
  },
  "chunking_settings": {
    "strategy": "sentence",
    "max_chunk_size": 250,
    "sentence_overlap": 1
  }
}




















Create a Mistral inference endpoint Generally available; Added in 8.15.0

PUT /_inference/{task_type}/{mistral_inference_id}

Creates an inference endpoint to perform an inference task with the mistral service.

Required authorization

  • Cluster privileges: manage_inference

Path parameters

  • task_type string Required

    The task type. The only valid task type for the model to perform is text_embedding.

    Value is text_embedding.

  • mistral_inference_id string Required

    The unique identifier of the inference endpoint.

application/json

Body

  • chunking_settings object

    Chunking configuration object

    Hide chunking_settings attributes Show chunking_settings attributes object
    • max_chunk_size number

      The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

    • overlap number

      The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

    • sentence_overlap number

      The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

    • strategy string

      The chunking strategy: sentence or word.

  • service string Required

    Value is mistral.

  • service_settings object Required
    Hide service_settings attributes Show service_settings attributes object
    • api_key string Required

      A valid API key of your Mistral account. You can find your Mistral API keys or you can create a new one on the API Keys page.

      IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key. After creating the inference model, you cannot change the associated API key. If you want to use a different API key, delete the inference model and recreate it with the same name and the updated API key.

      External documentation
    • max_input_tokens number

      The maximum number of tokens per input before chunking occurs.

    • model string Required

      The name of the model to use for the inference task. Refer to the Mistral models documentation for the list of available text embedding models.

      External documentation
    • rate_limit object
      Hide rate_limit attribute Show rate_limit attribute object
      • requests_per_minute number

        The number of requests allowed per minute.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • chunking_settings object

      Chunking configuration object

      Hide chunking_settings attributes Show chunking_settings attributes object
      • max_chunk_size number

        The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

      • overlap number

        The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

      • sentence_overlap number

        The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

      • strategy string

        The chunking strategy: sentence or word.

    • service string Required

      The service type

    • service_settings object Required
    • task_settings object
    • inference_id string Required

      The inference Id

    • task_type string Required

      Value is text_embedding.

PUT /_inference/{task_type}/{mistral_inference_id}
curl \
 --request PUT 'http://api.example.com/_inference/{task_type}/{mistral_inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"service\": \"mistral\",\n  \"service_settings\": {\n    \"api_key\": \"Mistral-API-Key\",\n    \"model\": \"mistral-embed\" \n  }\n}"'
Request example
Run `PUT _inference/text_embedding/mistral-embeddings-test` to create a Mistral inference endpoint that performs a text embedding task.
{
  "service": "mistral",
  "service_settings": {
    "api_key": "Mistral-API-Key",
    "model": "mistral-embed" 
  }
}
















Perform sparse embedding inference on the service Generally available; Added in 8.11.0

POST /_inference/sparse_embedding/{inference_id}

Path parameters

  • inference_id string Required

    The inference Id

Query parameters

  • timeout string

    Specifies the amount of time to wait for the inference request to complete.

    Values are -1 or 0.

application/json

Body

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • sparse_embedding array[object] Required
      Hide sparse_embedding attribute Show sparse_embedding attribute object
      • embedding object Required

        Sparse Embedding tokens are represented as a dictionary of string to double.

        Hide embedding attribute Show embedding attribute object
        • * number Additional properties
POST /_inference/sparse_embedding/{inference_id}
curl \
 --request POST 'http://api.example.com/_inference/sparse_embedding/{inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"input\": \"The sky above the port was the color of television tuned to a dead channel.\"\n}"'
Request example
Run `POST _inference/sparse_embedding/my-elser-model` to perform sparse embedding on the example sentence.
{
  "input": "The sky above the port was the color of television tuned to a dead channel."
}
Response examples (200)
An abbreviated response from `POST _inference/sparse_embedding/my-elser-model`.
{
  "sparse_embedding": [
    {
      "port": 2.1259406,
      "sky": 1.7073475,
      "color": 1.6922266,
      "dead": 1.6247464,
      "television": 1.3525393,
      "above": 1.2425821,
      "tuned": 1.1440028,
      "colors": 1.1218185,
      "tv": 1.0111054,
      "ports": 1.0067928,
      "poem": 1.0042328,
      "channel": 0.99471164,
      "tune": 0.96235967,
      "scene": 0.9020516
    }
  ]
}

Perform streaming inference Generally available; Added in 8.16.0

POST /_inference/completion/{inference_id}/_stream

Get real-time responses for completion tasks by delivering answers incrementally, reducing response times during computation. This API works only with the completion task type.

IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.

This API requires the monitor_inference cluster privilege (the built-in inference_admin and inference_user roles grant this privilege). You must use a client that supports streaming.

Required authorization

  • Cluster privileges: monitor_inference

Path parameters

  • inference_id string Required

    The unique identifier for the inference endpoint.

application/json

Body

  • input string | array[string] Required

    The text on which you want to perform the inference task. It can be a single string or an array.

    NOTE: Inference endpoints for the completion task type currently only support a single string as input.

  • task_settings object

Responses

  • 200 application/json
POST /_inference/completion/{inference_id}/_stream
curl \
 --request POST 'http://api.example.com/_inference/completion/{inference_id}/_stream' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"input\": \"What is Elastic?\"\n}"'
Request example
Run `POST _inference/completion/openai-completion/_stream` to perform a completion on the example question with streaming.
{
  "input": "What is Elastic?"
}

Perform text embedding inference on the service Generally available; Added in 8.11.0

POST /_inference/text_embedding/{inference_id}

Path parameters

  • inference_id string Required

    The inference Id

Query parameters

  • timeout string

    Specifies the amount of time to wait for the inference request to complete.

    Values are -1 or 0.

application/json

Body

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • text_embedding_bytes array[object]

      The text embedding result object for byte representation

      Hide text_embedding_bytes attribute Show text_embedding_bytes attribute object
      • embedding array[number] Required

        Text Embedding results containing bytes are represented as Dense Vectors of bytes.

    • text_embedding array[object]

      The text embedding result object

      Hide text_embedding attribute Show text_embedding attribute object
      • embedding array[number] Required

        Text Embedding results are represented as Dense Vectors of floats.

POST /_inference/text_embedding/{inference_id}
curl \
 --request POST 'http://api.example.com/_inference/text_embedding/{inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"input\": \"The sky above the port was the color of television tuned to a dead channel.\",\n  \"task_settings\": {\n    \"input_type\": \"ingest\"\n  }\n}"'
Request example
Run `POST _inference/text_embedding/my-cohere-endpoint` to perform text embedding on the example sentence using the Cohere integration,
{
  "input": "The sky above the port was the color of television tuned to a dead channel.",
  "task_settings": {
    "input_type": "ingest"
  }
}
Response examples (200)
An abbreviated response from `POST _inference/text_embedding/my-cohere-endpoint`.
{
  "text_embedding": [
    {
      "embedding": [
        {
          0.018569946,
          -0.036895752,
          0.01486969,
          -0.0045204163,
          -0.04385376,
          0.0075950623,
          0.04260254,
          -0.004005432,
          0.007865906,
          0.030792236,
          -0.050476074,
          0.011795044,
          -0.011642456,
          -0.010070801
        }
      ]
    }
  ]
}

Update an inference endpoint Generally available; Added in 8.17.0

PUT /_inference/{inference_id}/_update

Modify task_settings, secrets (within service_settings), or num_allocations for an inference endpoint, depending on the specific endpoint service and task_type.

IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.

Required authorization

  • Cluster privileges: manage_inference

Path parameters

  • inference_id string Required

    The unique identifier of the inference endpoint.

application/json

Body Required

  • chunking_settings object

    Chunking configuration object

    Hide chunking_settings attributes Show chunking_settings attributes object
    • max_chunk_size number

      The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

    • overlap number

      The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

    • sentence_overlap number

      The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

    • strategy string

      The chunking strategy: sentence or word.

  • service string Required

    The service type

  • service_settings object Required
  • task_settings object

Responses

  • 200 application/json
    Hide response attributes Show response attributes object

    Represents an inference endpoint as returned by the GET API

    • chunking_settings object

      Chunking configuration object

      Hide chunking_settings attributes Show chunking_settings attributes object
      • max_chunk_size number

        The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

      • overlap number

        The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

      • sentence_overlap number

        The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

      • strategy string

        The chunking strategy: sentence or word.

    • service string Required

      The service type

    • service_settings object Required
    • task_settings object
    • inference_id string Required

      The inference Id

    • task_type string Required

      Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.

PUT /_inference/{inference_id}/_update
curl \
 --request PUT 'http://api.example.com/_inference/{inference_id}/_update' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n \"service_settings\": {\n   \"api_key\": \"\u003cAPI_KEY\u003e\"\n }\n}"'
Request example
An example body for a `PUT _inference/my-inference-endpoint/_update` request.
{
 "service_settings": {
   "api_key": "<API_KEY>"
 }
}




Info

Get cluster info Generally available

GET /

Get basic build, version, and cluster information.

Required authorization

  • Cluster privileges: monitor

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • cluster_name string Required
    • cluster_uuid string Required
    • name string Required
    • tagline string Required
    • version object Required
      Hide version attributes Show version attributes object
      • build_date string | number Required

        A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

        One of:
      • build_flavor string Required

        The build flavor. For example, default.

      • build_hash string Required

        The Elasticsearch Git commit's SHA hash.

      • build_snapshot boolean Required

        Indicates whether the Elasticsearch build was a snapshot.

      • build_type string Required

        The build type that corresponds to how Elasticsearch was installed. For example, docker, rpm, or tar.

      • lucene_version string Required
      • minimum_index_compatibility_version string Required
      • minimum_wire_compatibility_version string Required
      • number string Required

        The Elasticsearch version number.

GET /
curl \
 --request GET 'http://api.example.com/' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /`s.
{
  "name": "instance-0000000000",
  "cluster_name": "my_test_cluster",
  "cluster_uuid": "5QaxoN0pRZuOmWSxstBBwQ",
  "version": {
    "build_date": "2024-02-01T13:07:13.727175297Z",
    "minimum_wire_compatibility_version": "7.17.0",
    "build_hash": "6185ba65d27469afabc9bc951cded6c17c21e3f3",
    "number": "8.12.1",
    "lucene_version": "9.9.2",
    "minimum_index_compatibility_version": "7.0.0",
    "build_flavor": "default",
    "build_snapshot": false,
    "build_type": "docker"
  },
  "tagline": "You Know, for Search"
}





























Create or update a pipeline Generally available; Added in 5.0.0

PUT /_ingest/pipeline/{id}

Changes made using this API take effect immediately.

External documentation

Path parameters

  • id string Required

    ID of the ingest pipeline to create or update.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are -1 or 0.

  • if_version number

    Required version for optimistic concurrency control for pipeline updates

application/json

Body Required

  • _meta object
    Hide _meta attribute Show _meta attribute object
    • * object Additional properties
  • description string

    Description of the ingest pipeline.

  • on_failure array[object]

    Processors to run immediately after a processor failure. Each processor supports a processor-level on_failure value. If a processor without an on_failure value fails, Elasticsearch uses this pipeline-level parameter as a fallback. The processors in this parameter run sequentially in the order specified. Elasticsearch will not attempt to run the pipeline's remaining processors.

    Hide on_failure attributes Show on_failure attributes object
    • append object
      Hide append attributes Show append attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • value object | array[object] Required

        The value to be appended. Supports template snippets.

      • allow_duplicates boolean

        If false, the processor does not append values already present in the field.

    • attachment object
      Hide attachment attributes Show attachment attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • indexed_chars number

        The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

      • indexed_chars_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • properties array[string]

        Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • remove_binary boolean

        If true, the binary field will be removed from the document

      • resource_name string

        Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

    • bytes object
      Hide bytes attributes Show bytes attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • circle object
      Hide circle attributes Show circle attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • error_distance number Required

        The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • shape_type string Required

        Values are geo_shape or shape.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • community_id object
      Hide community_id attributes Show community_id attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • source_ip string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • source_port string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • destination_ip string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • destination_port string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • iana_number string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • icmp_type string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • icmp_code string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • transport string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • seed number

        Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

      • ignore_missing boolean

        If true and any required fields are missing, the processor quietly exits without modifying the document.

    • convert object
      Hide convert attributes Show convert attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • type string Required

        Values are integer, long, double, float, boolean, ip, string, or auto.

    • csv object
      Hide csv attributes Show csv attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • empty_value object

        Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • quote string

        Quote used in CSV, has to be single character string.

      • separator string

        Separator used in CSV, has to be single character string.

      • target_fields string | array[string] Required
      • trim boolean

        Trim whitespaces in unquoted fields.

    • date object
      Hide date attributes Show date attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • formats array[string] Required

        An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

      • locale string

        The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • timezone string

        The timezone to use when parsing the date. Supports template snippets.

      • output_format string

        The format to use when writing the date to target_field. Must be a valid java time pattern.

    • date_index_name object
      Hide date_index_name attributes Show date_index_name attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • date_formats array[string] Required

        An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

      • date_rounding string Required

        How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • index_name_format string

        The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

      • index_name_prefix string

        A prefix of the index name to be prepended before the printed date. Supports template snippets.

      • locale string

        The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

      • timezone string

        The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

    • dissect object
      Hide dissect attributes Show dissect attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • append_separator string

        The character(s) that separate the appended fields.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • pattern string Required

        The pattern to apply to the field.

    • dot_expander object
      Hide dot_expander attributes Show dot_expander attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • override boolean

        Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

      • path string

        The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

    • drop object
      Hide drop attributes Show drop attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

    • enrich object
      Hide enrich attributes Show enrich attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • max_matches number

        The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

      • override boolean

        If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

      • policy_name string Required

        The name of the enrich policy to use.

      • shape_relation string

        Values are intersects, disjoint, within, or contains.

      • target_field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • fail object
      Hide fail attributes Show fail attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • message string Required

        The error message thrown by the processor. Supports template snippets.

    • fingerprint object
      Hide fingerprint attributes Show fingerprint attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • fields string | array[string] Required
      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • salt string

        Salt value for the hash function.

      • method string

        Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

      • ignore_missing boolean

        If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

    • foreach object
      Hide foreach attributes Show foreach attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true, the processor silently exits without changing the document if the field is null or missing.

      • processor object Required
    • ip_location object
      Hide ip_location attributes Show ip_location attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • database_file string

        The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • first_only boolean

        If true, only the first found IP location data will be returned, even if the field contains an array.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • properties array[string]

        Controls what properties are added to the target_field based on the IP location lookup.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • download_database_on_pipeline_creation boolean

        If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

    • geo_grid object
      Hide geo_grid attributes Show geo_grid attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        The field to interpret as a geo-tile.= The field format is determined by the tile_type.

      • tile_type string Required

        Values are geotile, geohex, or geohash.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • parent_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • children_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • non_children_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • precision_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • target_format string

        Values are geojson or wkt.

    • geoip object
      Hide geoip attributes Show geoip attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • database_file string

        The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • first_only boolean

        If true, only the first found geoip data will be returned, even if the field contains an array.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • properties array[string]

        Controls what properties are added to the target_field based on the geoip lookup.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • download_database_on_pipeline_creation boolean

        If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

    • grok object
      Hide grok attributes Show grok attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • ecs_compatibility string

        Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • pattern_definitions object

        A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

        Hide pattern_definitions attribute Show pattern_definitions attribute object
        • * string Additional properties
      • patterns array[string] Required

        An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

      • trace_match boolean

        When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

    • gsub object
      Hide gsub attributes Show gsub attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • pattern string Required

        The pattern to be replaced.

      • replacement string Required

        The string to replace the matching patterns with.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • html_strip object
      Hide html_strip attributes Show html_strip attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document,

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • inference object
      Hide inference attributes Show inference attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • model_id string Required
      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • field_map object

        Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

        Hide field_map attribute Show field_map attribute object
        • * object Additional properties
      • inference_config object
        Hide inference_config attributes Show inference_config attributes object
        • regression object
          Hide regression attributes Show regression attributes object
          • results_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • num_top_feature_importance_values number

            Specifies the maximum number of feature importance values per document.

        • classification object
          Hide classification attributes Show classification attributes object
          • num_top_classes number

            Specifies the number of top class predictions to return.

          • num_top_feature_importance_values number

            Specifies the maximum number of feature importance values per document.

          • results_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • top_classes_results_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • prediction_field_type string

            Specifies the type of the predicted field to write. Valid values are: string, number, boolean.

      • input_output object | array[object]

        Input fields for inference and output (destination) fields for the inference results. This option is incompatible with the target_field and field_map options.

        One of:
        Hide attributes Show attributes
        • input_field string Required
        • output_field string Required
      • ignore_missing boolean

        If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

    • join object
      Hide join attributes Show join attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • separator string Required

        The separator character.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • json object
      Hide json attributes Show json attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • add_to_root boolean

        Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

      • add_to_root_conflict_strategy string

        Values are replace or merge.

      • allow_duplicate_keys boolean

        When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • kv object
      Hide kv attributes Show kv attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • exclude_keys array[string]

        List of keys to exclude from document.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • field_split string Required

        Regex pattern to use for splitting key-value pairs.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • include_keys array[string]

        List of keys to filter and insert into document. Defaults to including all keys.

      • prefix string

        Prefix to be added to extracted keys.

      • strip_brackets boolean

        If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • trim_key string

        String of characters to trim from extracted keys.

      • trim_value string

        String of characters to trim from extracted values.

      • value_split string Required

        Regex pattern to use for splitting the key from the value within a key-value pair.

    • lowercase object
      Hide lowercase attributes Show lowercase attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • network_direction object
      Hide network_direction attributes Show network_direction attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • source_ip string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • destination_ip string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • internal_networks array[string]

        List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

      • internal_networks_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and any required fields are missing, the processor quietly exits without modifying the document.

    • pipeline object
      Hide pipeline attributes Show pipeline attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • name string Required
      • ignore_missing_pipeline boolean

        Whether to ignore missing pipelines instead of failing.

    • redact object
      Hide redact attributes Show redact attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • patterns array[string] Required

        A list of grok expressions to match and redact named captures with

      • pattern_definitions object
        Hide pattern_definitions attribute Show pattern_definitions attribute object
        • * string Additional properties
      • prefix string

        Start a redacted section with this token

      • suffix string

        End a redacted section with this token

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • skip_if_unlicensed boolean

        If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

      • trace_redact boolean Generally available; Added in 8.16.0

        If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

    • registered_domain object
      Hide registered_domain attributes Show registered_domain attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and any required fields are missing, the processor quietly exits without modifying the document.

    • remove object
      Hide remove attributes Show remove attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string | array[string] Required
      • keep string | array[string]
      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

    • rename object
      Hide rename attributes Show rename attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • target_field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • reroute object
      Hide reroute attributes Show reroute attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • destination string

        A static value for the target. Can’t be set when the dataset or namespace option is set.

      • dataset string | array[string]

        Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.

        Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

        default {{data_stream.dataset}}

      • namespace string | array[string]

        Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.

        Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

        default {{data_stream.namespace}}

    • script object
      Hide script attributes Show script attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • id string
      • lang string

        Script language.

      • params object

        Object containing parameters for the script.

        Hide params attribute Show params attribute object
        • * object Additional properties
      • source string

        Inline script. If no id is specified, this parameter is required.

    • set object
      Hide set attributes Show set attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • copy_from string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_empty_value boolean

        If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

      • media_type string

        The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

      • override boolean

        If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

      • value object

        The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

    • set_security_user object
      Hide set_security_user attributes Show set_security_user attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • properties array[string]

        Controls what user related properties are added to the field.

    • sort object
      Hide sort attributes Show sort attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • order string

        Values are asc or desc.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • split object
      Hide split attributes Show split attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • preserve_trailing boolean

        Preserves empty trailing fields, if any.

      • separator string Required

        A regex which matches the separator, for example, , or \s+.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • terminate object
      Hide terminate attributes Show terminate attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

    • trim object
      Hide trim attributes Show trim attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • uppercase object
      Hide uppercase attributes Show uppercase attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • urldecode object
      Hide urldecode attributes Show urldecode attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • uri_parts object
      Hide uri_parts attributes Show uri_parts attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • keep_original boolean

        If true, the processor copies the unparsed URI to <target_field>.original.

      • remove_if_successful boolean

        If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • user_agent object
      Hide user_agent attributes Show user_agent attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • regex_file string

        The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • properties array[string]

        Controls what properties are added to target_field.

        Values are name, os, device, original, or version.

      • extract_device_type boolean Beta; Added in 8.9.0

        Extracts device type from the user agent string on a best-effort basis.

  • processors array[object]

    Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.

    Hide processors attributes Show processors attributes object
    • append object
      Hide append attributes Show append attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • value object | array[object] Required

        The value to be appended. Supports template snippets.

      • allow_duplicates boolean

        If false, the processor does not append values already present in the field.

    • attachment object
      Hide attachment attributes Show attachment attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • indexed_chars number

        The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

      • indexed_chars_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • properties array[string]

        Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • remove_binary boolean

        If true, the binary field will be removed from the document

      • resource_name string

        Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

    • bytes object
      Hide bytes attributes Show bytes attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • circle object
      Hide circle attributes Show circle attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • error_distance number Required

        The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • shape_type string Required

        Values are geo_shape or shape.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • community_id object
      Hide community_id attributes Show community_id attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • source_ip string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • source_port string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • destination_ip string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • destination_port string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • iana_number string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • icmp_type string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • icmp_code string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • transport string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • seed number

        Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

      • ignore_missing boolean

        If true and any required fields are missing, the processor quietly exits without modifying the document.

    • convert object
      Hide convert attributes Show convert attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • type string Required

        Values are integer, long, double, float, boolean, ip, string, or auto.

    • csv object
      Hide csv attributes Show csv attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • empty_value object

        Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • quote string

        Quote used in CSV, has to be single character string.

      • separator string

        Separator used in CSV, has to be single character string.

      • target_fields string | array[string] Required
      • trim boolean

        Trim whitespaces in unquoted fields.

    • date object
      Hide date attributes Show date attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • formats array[string] Required

        An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

      • locale string

        The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • timezone string

        The timezone to use when parsing the date. Supports template snippets.

      • output_format string

        The format to use when writing the date to target_field. Must be a valid java time pattern.

    • date_index_name object
      Hide date_index_name attributes Show date_index_name attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • date_formats array[string] Required

        An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

      • date_rounding string Required

        How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • index_name_format string

        The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

      • index_name_prefix string

        A prefix of the index name to be prepended before the printed date. Supports template snippets.

      • locale string

        The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

      • timezone string

        The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

    • dissect object
      Hide dissect attributes Show dissect attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • append_separator string

        The character(s) that separate the appended fields.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • pattern string Required

        The pattern to apply to the field.

    • dot_expander object
      Hide dot_expander attributes Show dot_expander attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • override boolean

        Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

      • path string

        The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

    • drop object
      Hide drop attributes Show drop attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

    • enrich object
      Hide enrich attributes Show enrich attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • max_matches number

        The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

      • override boolean

        If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

      • policy_name string Required

        The name of the enrich policy to use.

      • shape_relation string

        Values are intersects, disjoint, within, or contains.

      • target_field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • fail object
      Hide fail attributes Show fail attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • message string Required

        The error message thrown by the processor. Supports template snippets.

    • fingerprint object
      Hide fingerprint attributes Show fingerprint attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • fields string | array[string] Required
      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • salt string

        Salt value for the hash function.

      • method string

        Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

      • ignore_missing boolean

        If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

    • foreach object
      Hide foreach attributes Show foreach attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true, the processor silently exits without changing the document if the field is null or missing.

      • processor object Required
    • ip_location object
      Hide ip_location attributes Show ip_location attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • database_file string

        The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • first_only boolean

        If true, only the first found IP location data will be returned, even if the field contains an array.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • properties array[string]

        Controls what properties are added to the target_field based on the IP location lookup.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • download_database_on_pipeline_creation boolean

        If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

    • geo_grid object
      Hide geo_grid attributes Show geo_grid attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        The field to interpret as a geo-tile.= The field format is determined by the tile_type.

      • tile_type string Required

        Values are geotile, geohex, or geohash.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • parent_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • children_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • non_children_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • precision_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • target_format string

        Values are geojson or wkt.

    • geoip object
      Hide geoip attributes Show geoip attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • database_file string

        The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • first_only boolean

        If true, only the first found geoip data will be returned, even if the field contains an array.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • properties array[string]

        Controls what properties are added to the target_field based on the geoip lookup.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • download_database_on_pipeline_creation boolean

        If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

    • grok object
      Hide grok attributes Show grok attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • ecs_compatibility string

        Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • pattern_definitions object

        A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

        Hide pattern_definitions attribute Show pattern_definitions attribute object
        • * string Additional properties
      • patterns array[string] Required

        An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

      • trace_match boolean

        When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

    • gsub object
      Hide gsub attributes Show gsub attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • pattern string Required

        The pattern to be replaced.

      • replacement string Required

        The string to replace the matching patterns with.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • html_strip object
      Hide html_strip attributes Show html_strip attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document,

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • inference object
      Hide inference attributes Show inference attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • model_id string Required
      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • field_map object

        Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

        Hide field_map attribute Show field_map attribute object
        • * object Additional properties
      • inference_config object
        Hide inference_config attributes Show inference_config attributes object
        • regression object
          Hide regression attributes Show regression attributes object
          • results_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • num_top_feature_importance_values number

            Specifies the maximum number of feature importance values per document.

        • classification object
          Hide classification attributes Show classification attributes object
          • num_top_classes number

            Specifies the number of top class predictions to return.

          • num_top_feature_importance_values number

            Specifies the maximum number of feature importance values per document.

          • results_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • top_classes_results_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • prediction_field_type string

            Specifies the type of the predicted field to write. Valid values are: string, number, boolean.

      • input_output object | array[object]

        Input fields for inference and output (destination) fields for the inference results. This option is incompatible with the target_field and field_map options.

        One of:
        Hide attributes Show attributes
        • input_field string Required
        • output_field string Required
      • ignore_missing boolean

        If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

    • join object
      Hide join attributes Show join attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • separator string Required

        The separator character.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • json object
      Hide json attributes Show json attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • add_to_root boolean

        Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

      • add_to_root_conflict_strategy string

        Values are replace or merge.

      • allow_duplicate_keys boolean

        When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • kv object
      Hide kv attributes Show kv attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • exclude_keys array[string]

        List of keys to exclude from document.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • field_split string Required

        Regex pattern to use for splitting key-value pairs.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • include_keys array[string]

        List of keys to filter and insert into document. Defaults to including all keys.

      • prefix string

        Prefix to be added to extracted keys.

      • strip_brackets boolean

        If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • trim_key string

        String of characters to trim from extracted keys.

      • trim_value string

        String of characters to trim from extracted values.

      • value_split string Required

        Regex pattern to use for splitting the key from the value within a key-value pair.

    • lowercase object
      Hide lowercase attributes Show lowercase attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • network_direction object
      Hide network_direction attributes Show network_direction attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • source_ip string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • destination_ip string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • internal_networks array[string]

        List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

      • internal_networks_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and any required fields are missing, the processor quietly exits without modifying the document.

    • pipeline object
      Hide pipeline attributes Show pipeline attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • name string Required
      • ignore_missing_pipeline boolean

        Whether to ignore missing pipelines instead of failing.

    • redact object
      Hide redact attributes Show redact attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • patterns array[string] Required

        A list of grok expressions to match and redact named captures with

      • pattern_definitions object
        Hide pattern_definitions attribute Show pattern_definitions attribute object
        • * string Additional properties
      • prefix string

        Start a redacted section with this token

      • suffix string

        End a redacted section with this token

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • skip_if_unlicensed boolean

        If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

      • trace_redact boolean Generally available; Added in 8.16.0

        If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

    • registered_domain object
      Hide registered_domain attributes Show registered_domain attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and any required fields are missing, the processor quietly exits without modifying the document.

    • remove object
      Hide remove attributes Show remove attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string | array[string] Required
      • keep string | array[string]
      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

    • rename object
      Hide rename attributes Show rename attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • target_field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • reroute object
      Hide reroute attributes Show reroute attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • destination string

        A static value for the target. Can’t be set when the dataset or namespace option is set.

      • dataset string | array[string]

        Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.

        Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

        default {{data_stream.dataset}}

      • namespace string | array[string]

        Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.

        Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

        default {{data_stream.namespace}}

    • script object
      Hide script attributes Show script attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • id string
      • lang string

        Script language.

      • params object

        Object containing parameters for the script.

        Hide params attribute Show params attribute object
        • * object Additional properties
      • source string

        Inline script. If no id is specified, this parameter is required.

    • set object
      Hide set attributes Show set attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • copy_from string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_empty_value boolean

        If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

      • media_type string

        The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

      • override boolean

        If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

      • value object

        The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

    • set_security_user object
      Hide set_security_user attributes Show set_security_user attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • properties array[string]

        Controls what user related properties are added to the field.

    • sort object
      Hide sort attributes Show sort attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • order string

        Values are asc or desc.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • split object
      Hide split attributes Show split attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • preserve_trailing boolean

        Preserves empty trailing fields, if any.

      • separator string Required

        A regex which matches the separator, for example, , or \s+.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • terminate object
      Hide terminate attributes Show terminate attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

    • trim object
      Hide trim attributes Show trim attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • uppercase object
      Hide uppercase attributes Show uppercase attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • urldecode object
      Hide urldecode attributes Show urldecode attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • uri_parts object
      Hide uri_parts attributes Show uri_parts attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • keep_original boolean

        If true, the processor copies the unparsed URI to <target_field>.original.

      • remove_if_successful boolean

        If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • user_agent object
      Hide user_agent attributes Show user_agent attributes object
      • description string

        Description of the processor. Useful for describing the purpose of the processor or its configuration.

      • if string

        Conditionally execute the processor.

      • ignore_failure boolean

        Ignore failures for the processor.

      • on_failure array[object]

        Handle failures for the processor.

      • tag string

        Identifier for the processor. Useful for debugging and metrics.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • ignore_missing boolean

        If true and field does not exist, the processor quietly exits without modifying the document.

      • regex_file string

        The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • properties array[string]

        Controls what properties are added to target_field.

        Values are name, os, device, original, or version.

      • extract_device_type boolean Beta; Added in 8.9.0

        Extracts device type from the user agent string on a best-effort basis.

  • version number
  • deprecated boolean

    Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_ingest/pipeline/{id}
curl \
 --request PUT 'http://api.example.com/_ingest/pipeline/{id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"description\" : \"My optional pipeline description\",\n  \"processors\" : [\n    {\n      \"set\" : {\n        \"description\" : \"My optional processor description\",\n        \"field\": \"my-keyword-field\",\n        \"value\": \"foo\"\n      }\n    }\n  ]\n}"'
{
  "description" : "My optional pipeline description",
  "processors" : [
    {
      "set" : {
        "description" : "My optional processor description",
        "field": "my-keyword-field",
        "value": "foo"
      }
    }
  ]
}
You can use the `_meta` parameter to add arbitrary metadata to a pipeline.
{
  "description" : "My optional pipeline description",
  "processors" : [
    {
      "set" : {
        "description" : "My optional processor description",
        "field": "my-keyword-field",
        "value": "foo"
      }
    }
  ],
  "_meta": {
    "reason": "set my-keyword-field to foo",
    "serialization": {
      "class": "MyPipeline",
      "id": 10
    }
  }
}




























Simulate a pipeline Generally available; Added in 5.0.0

POST /_ingest/pipeline/_simulate

Run an ingest pipeline against a set of provided documents. You can either specify an existing pipeline to use with the provided documents or supply a pipeline definition in the body of the request.

Required authorization

  • Cluster privileges: read_pipeline

Query parameters

  • verbose boolean

    If true, the response includes output data for each processor in the executed pipeline.

application/json

Body Required

  • docs array[object] Required

    Sample documents to test in the pipeline.

    Hide docs attributes Show docs attributes object
    • _id string
    • _index string
    • _source object Required

      JSON body for the document.

  • pipeline object Additional properties
    Hide pipeline attributes Show pipeline attributes object
    • description string

      Description of the ingest pipeline.

    • on_failure array[object]

      Processors to run immediately after a processor failure.

      Hide on_failure attributes Show on_failure attributes object
      • append object
        Hide append attributes Show append attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • value object | array[object] Required

          The value to be appended. Supports template snippets.

        • allow_duplicates boolean

          If false, the processor does not append values already present in the field.

      • attachment object
        Hide attachment attributes Show attachment attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • indexed_chars number

          The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

        • indexed_chars_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • remove_binary boolean

          If true, the binary field will be removed from the document

        • resource_name string

          Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

      • bytes object
        Hide bytes attributes Show bytes attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • circle object
        Hide circle attributes Show circle attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • error_distance number Required

          The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • shape_type string Required

          Values are geo_shape or shape.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • community_id object
        Hide community_id attributes Show community_id attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • source_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • source_port string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_port string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • iana_number string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • icmp_type string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • icmp_code string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • transport string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • seed number

          Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • convert object
        Hide convert attributes Show convert attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • type string Required

          Values are integer, long, double, float, boolean, ip, string, or auto.

      • csv object
        Hide csv attributes Show csv attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • empty_value object

          Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • quote string

          Quote used in CSV, has to be single character string.

        • separator string

          Separator used in CSV, has to be single character string.

        • target_fields string | array[string] Required
        • trim boolean

          Trim whitespaces in unquoted fields.

      • date object
        Hide date attributes Show date attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • formats array[string] Required

          An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

        • locale string

          The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • timezone string

          The timezone to use when parsing the date. Supports template snippets.

        • output_format string

          The format to use when writing the date to target_field. Must be a valid java time pattern.

      • date_index_name object
        Hide date_index_name attributes Show date_index_name attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • date_formats array[string] Required

          An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

        • date_rounding string Required

          How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • index_name_format string

          The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

        • index_name_prefix string

          A prefix of the index name to be prepended before the printed date. Supports template snippets.

        • locale string

          The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

        • timezone string

          The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

      • dissect object
        Hide dissect attributes Show dissect attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • append_separator string

          The character(s) that separate the appended fields.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern string Required

          The pattern to apply to the field.

      • dot_expander object
        Hide dot_expander attributes Show dot_expander attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • override boolean

          Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

        • path string

          The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

      • drop object
        Hide drop attributes Show drop attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

      • enrich object
        Hide enrich attributes Show enrich attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • max_matches number

          The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

        • override boolean

          If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        • policy_name string Required

          The name of the enrich policy to use.

        • shape_relation string

          Values are intersects, disjoint, within, or contains.

        • target_field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • fail object
        Hide fail attributes Show fail attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • message string Required

          The error message thrown by the processor. Supports template snippets.

      • fingerprint object
        Hide fingerprint attributes Show fingerprint attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • fields string | array[string] Required
        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • salt string

          Salt value for the hash function.

        • method string

          Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

        • ignore_missing boolean

          If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

      • foreach object
        Hide foreach attributes Show foreach attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true, the processor silently exits without changing the document if the field is null or missing.

        • processor object Required
      • ip_location object
        Hide ip_location attributes Show ip_location attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • database_file string

          The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • first_only boolean

          If true, only the first found IP location data will be returned, even if the field contains an array.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • properties array[string]

          Controls what properties are added to the target_field based on the IP location lookup.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • download_database_on_pipeline_creation boolean

          If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

      • geo_grid object
        Hide geo_grid attributes Show geo_grid attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          The field to interpret as a geo-tile.= The field format is determined by the tile_type.

        • tile_type string Required

          Values are geotile, geohex, or geohash.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • parent_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • children_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • non_children_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • precision_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_format string

          Values are geojson or wkt.

      • geoip object
        Hide geoip attributes Show geoip attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • database_file string

          The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • first_only boolean

          If true, only the first found geoip data will be returned, even if the field contains an array.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • properties array[string]

          Controls what properties are added to the target_field based on the geoip lookup.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • download_database_on_pipeline_creation boolean

          If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

      • grok object
        Hide grok attributes Show grok attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • ecs_compatibility string

          Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern_definitions object

          A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          Hide pattern_definitions attribute Show pattern_definitions attribute object
          • * string Additional properties
        • patterns array[string] Required

          An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

        • trace_match boolean

          When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

      • gsub object
        Hide gsub attributes Show gsub attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern string Required

          The pattern to be replaced.

        • replacement string Required

          The string to replace the matching patterns with.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • html_strip object
        Hide html_strip attributes Show html_strip attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document,

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • inference object
        Hide inference attributes Show inference attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • model_id string Required
        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field_map object

          Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          Hide field_map attribute Show field_map attribute object
          • * object Additional properties
        • inference_config object
          Hide inference_config attributes Show inference_config attributes object
          • regression object
          • classification object
        • input_output object | array[object]

          Input fields for inference and output (destination) fields for the inference results. This option is incompatible with the target_field and field_map options.

        • ignore_missing boolean

          If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

      • join object
        Hide join attributes Show join attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • separator string Required

          The separator character.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • json object
        Hide json attributes Show json attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • add_to_root boolean

          Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

        • add_to_root_conflict_strategy string

          Values are replace or merge.

        • allow_duplicate_keys boolean

          When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • kv object
        Hide kv attributes Show kv attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • exclude_keys array[string]

          List of keys to exclude from document.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field_split string Required

          Regex pattern to use for splitting key-value pairs.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • include_keys array[string]

          List of keys to filter and insert into document. Defaults to including all keys.

        • prefix string

          Prefix to be added to extracted keys.

        • strip_brackets boolean

          If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • trim_key string

          String of characters to trim from extracted keys.

        • trim_value string

          String of characters to trim from extracted values.

        • value_split string Required

          Regex pattern to use for splitting the key from the value within a key-value pair.

      • lowercase object
        Hide lowercase attributes Show lowercase attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • network_direction object
        Hide network_direction attributes Show network_direction attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • source_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • internal_networks array[string]

          List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

        • internal_networks_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • pipeline object
        Hide pipeline attributes Show pipeline attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • name string Required
        • ignore_missing_pipeline boolean

          Whether to ignore missing pipelines instead of failing.

      • redact object
        Hide redact attributes Show redact attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • patterns array[string] Required

          A list of grok expressions to match and redact named captures with

        • pattern_definitions object
          Hide pattern_definitions attribute Show pattern_definitions attribute object
          • * string Additional properties
        • prefix string

          Start a redacted section with this token

        • suffix string

          End a redacted section with this token

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • skip_if_unlicensed boolean

          If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

        • trace_redact boolean Generally available; Added in 8.16.0

          If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

      • registered_domain object
        Hide registered_domain attributes Show registered_domain attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • remove object
        Hide remove attributes Show remove attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string | array[string] Required
        • keep string | array[string]
        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • rename object
        Hide rename attributes Show rename attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • reroute object
        Hide reroute attributes Show reroute attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • destination string

          A static value for the target. Can’t be set when the dataset or namespace option is set.

        • dataset string | array[string]

          Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.

          Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

          default {{data_stream.dataset}}

        • namespace string | array[string]

          Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.

          Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

          default {{data_stream.namespace}}

      • script object
        Hide script attributes Show script attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • id string
        • lang string

          Script language.

        • params object

          Object containing parameters for the script.

          Hide params attribute Show params attribute object
          • * object Additional properties
        • source string

          Inline script. If no id is specified, this parameter is required.

      • set object
        Hide set attributes Show set attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • copy_from string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_empty_value boolean

          If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

        • media_type string

          The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

        • override boolean

          If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        • value object

          The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

      • set_security_user object
        Hide set_security_user attributes Show set_security_user attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Controls what user related properties are added to the field.

      • sort object
        Hide sort attributes Show sort attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • order string

          Values are asc or desc.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • split object
        Hide split attributes Show split attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • preserve_trailing boolean

          Preserves empty trailing fields, if any.

        • separator string Required

          A regex which matches the separator, for example, , or \s+.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • terminate object
        Hide terminate attributes Show terminate attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

      • trim object
        Hide trim attributes Show trim attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • uppercase object
        Hide uppercase attributes Show uppercase attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • urldecode object
        Hide urldecode attributes Show urldecode attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • uri_parts object
        Hide uri_parts attributes Show uri_parts attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • keep_original boolean

          If true, the processor copies the unparsed URI to <target_field>.original.

        • remove_if_successful boolean

          If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • user_agent object
        Hide user_agent attributes Show user_agent attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • regex_file string

          The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Controls what properties are added to target_field.

          Values are name, os, device, original, or version.

        • extract_device_type boolean Beta; Added in 8.9.0

          Extracts device type from the user agent string on a best-effort basis.

    • processors array[object]

      Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.

      Hide processors attributes Show processors attributes object
      • append object
        Hide append attributes Show append attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • value object | array[object] Required

          The value to be appended. Supports template snippets.

        • allow_duplicates boolean

          If false, the processor does not append values already present in the field.

      • attachment object
        Hide attachment attributes Show attachment attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • indexed_chars number

          The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

        • indexed_chars_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • remove_binary boolean

          If true, the binary field will be removed from the document

        • resource_name string

          Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

      • bytes object
        Hide bytes attributes Show bytes attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • circle object
        Hide circle attributes Show circle attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • error_distance number Required

          The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • shape_type string Required

          Values are geo_shape or shape.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • community_id object
        Hide community_id attributes Show community_id attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • source_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • source_port string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_port string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • iana_number string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • icmp_type string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • icmp_code string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • transport string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • seed number

          Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • convert object
        Hide convert attributes Show convert attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • type string Required

          Values are integer, long, double, float, boolean, ip, string, or auto.

      • csv object
        Hide csv attributes Show csv attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • empty_value object

          Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • quote string

          Quote used in CSV, has to be single character string.

        • separator string

          Separator used in CSV, has to be single character string.

        • target_fields string | array[string] Required
        • trim boolean

          Trim whitespaces in unquoted fields.

      • date object
        Hide date attributes Show date attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • formats array[string] Required

          An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

        • locale string

          The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • timezone string

          The timezone to use when parsing the date. Supports template snippets.

        • output_format string

          The format to use when writing the date to target_field. Must be a valid java time pattern.

      • date_index_name object
        Hide date_index_name attributes Show date_index_name attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • date_formats array[string] Required

          An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

        • date_rounding string Required

          How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • index_name_format string

          The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

        • index_name_prefix string

          A prefix of the index name to be prepended before the printed date. Supports template snippets.

        • locale string

          The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

        • timezone string

          The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

      • dissect object
        Hide dissect attributes Show dissect attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • append_separator string

          The character(s) that separate the appended fields.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern string Required

          The pattern to apply to the field.

      • dot_expander object
        Hide dot_expander attributes Show dot_expander attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • override boolean

          Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

        • path string

          The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

      • drop object
        Hide drop attributes Show drop attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

      • enrich object
        Hide enrich attributes Show enrich attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • max_matches number

          The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

        • override boolean

          If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        • policy_name string Required

          The name of the enrich policy to use.

        • shape_relation string

          Values are intersects, disjoint, within, or contains.

        • target_field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • fail object
        Hide fail attributes Show fail attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • message string Required

          The error message thrown by the processor. Supports template snippets.

      • fingerprint object
        Hide fingerprint attributes Show fingerprint attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • fields string | array[string] Required
        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • salt string

          Salt value for the hash function.

        • method string

          Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

        • ignore_missing boolean

          If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

      • foreach object
        Hide foreach attributes Show foreach attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true, the processor silently exits without changing the document if the field is null or missing.

        • processor object Required
      • ip_location object
        Hide ip_location attributes Show ip_location attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • database_file string

          The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • first_only boolean

          If true, only the first found IP location data will be returned, even if the field contains an array.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • properties array[string]

          Controls what properties are added to the target_field based on the IP location lookup.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • download_database_on_pipeline_creation boolean

          If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

      • geo_grid object
        Hide geo_grid attributes Show geo_grid attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          The field to interpret as a geo-tile.= The field format is determined by the tile_type.

        • tile_type string Required

          Values are geotile, geohex, or geohash.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • parent_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • children_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • non_children_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • precision_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_format string

          Values are geojson or wkt.

      • geoip object
        Hide geoip attributes Show geoip attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • database_file string

          The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • first_only boolean

          If true, only the first found geoip data will be returned, even if the field contains an array.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • properties array[string]

          Controls what properties are added to the target_field based on the geoip lookup.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • download_database_on_pipeline_creation boolean

          If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

      • grok object
        Hide grok attributes Show grok attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • ecs_compatibility string

          Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern_definitions object

          A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          Hide pattern_definitions attribute Show pattern_definitions attribute object
          • * string Additional properties
        • patterns array[string] Required

          An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

        • trace_match boolean

          When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

      • gsub object
        Hide gsub attributes Show gsub attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern string Required

          The pattern to be replaced.

        • replacement string Required

          The string to replace the matching patterns with.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • html_strip object
        Hide html_strip attributes Show html_strip attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document,

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • inference object
        Hide inference attributes Show inference attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • model_id string Required
        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field_map object

          Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          Hide field_map attribute Show field_map attribute object
          • * object Additional properties
        • inference_config object
          Hide inference_config attributes Show inference_config attributes object
          • regression object
          • classification object
        • input_output object | array[object]

          Input fields for inference and output (destination) fields for the inference results. This option is incompatible with the target_field and field_map options.

        • ignore_missing boolean

          If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

      • join object
        Hide join attributes Show join attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • separator string Required

          The separator character.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • json object
        Hide json attributes Show json attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • add_to_root boolean

          Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

        • add_to_root_conflict_strategy string

          Values are replace or merge.

        • allow_duplicate_keys boolean

          When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • kv object
        Hide kv attributes Show kv attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • exclude_keys array[string]

          List of keys to exclude from document.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field_split string Required

          Regex pattern to use for splitting key-value pairs.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • include_keys array[string]

          List of keys to filter and insert into document. Defaults to including all keys.

        • prefix string

          Prefix to be added to extracted keys.

        • strip_brackets boolean

          If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • trim_key string

          String of characters to trim from extracted keys.

        • trim_value string

          String of characters to trim from extracted values.

        • value_split string Required

          Regex pattern to use for splitting the key from the value within a key-value pair.

      • lowercase object
        Hide lowercase attributes Show lowercase attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • network_direction object
        Hide network_direction attributes Show network_direction attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • source_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • internal_networks array[string]

          List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

        • internal_networks_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • pipeline object
        Hide pipeline attributes Show pipeline attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • name string Required
        • ignore_missing_pipeline boolean

          Whether to ignore missing pipelines instead of failing.

      • redact object
        Hide redact attributes Show redact attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • patterns array[string] Required

          A list of grok expressions to match and redact named captures with

        • pattern_definitions object
          Hide pattern_definitions attribute Show pattern_definitions attribute object
          • * string Additional properties
        • prefix string

          Start a redacted section with this token

        • suffix string

          End a redacted section with this token

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • skip_if_unlicensed boolean

          If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

        • trace_redact boolean Generally available; Added in 8.16.0

          If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

      • registered_domain object
        Hide registered_domain attributes Show registered_domain attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • remove object
        Hide remove attributes Show remove attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string | array[string] Required
        • keep string | array[string]
        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • rename object
        Hide rename attributes Show rename attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • reroute object
        Hide reroute attributes Show reroute attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • destination string

          A static value for the target. Can’t be set when the dataset or namespace option is set.

        • dataset string | array[string]

          Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.

          Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

          default {{data_stream.dataset}}

        • namespace string | array[string]

          Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.

          Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

          default {{data_stream.namespace}}

      • script object
        Hide script attributes Show script attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • id string
        • lang string

          Script language.

        • params object

          Object containing parameters for the script.

          Hide params attribute Show params attribute object
          • * object Additional properties
        • source string

          Inline script. If no id is specified, this parameter is required.

      • set object
        Hide set attributes Show set attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • copy_from string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_empty_value boolean

          If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

        • media_type string

          The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

        • override boolean

          If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        • value object

          The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

      • set_security_user object
        Hide set_security_user attributes Show set_security_user attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Controls what user related properties are added to the field.

      • sort object
        Hide sort attributes Show sort attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • order string

          Values are asc or desc.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • split object
        Hide split attributes Show split attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • preserve_trailing boolean

          Preserves empty trailing fields, if any.

        • separator string Required

          A regex which matches the separator, for example, , or \s+.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • terminate object
        Hide terminate attributes Show terminate attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

      • trim object
        Hide trim attributes Show trim attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • uppercase object
        Hide uppercase attributes Show uppercase attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • urldecode object
        Hide urldecode attributes Show urldecode attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • uri_parts object
        Hide uri_parts attributes Show uri_parts attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • keep_original boolean

          If true, the processor copies the unparsed URI to <target_field>.original.

        • remove_if_successful boolean

          If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • user_agent object
        Hide user_agent attributes Show user_agent attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • regex_file string

          The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Controls what properties are added to target_field.

          Values are name, os, device, original, or version.

        • extract_device_type boolean Beta; Added in 8.9.0

          Extracts device type from the user agent string on a best-effort basis.

    • version number
    • deprecated boolean

      Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.

    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required
      Hide docs attributes Show docs attributes object
      • doc object

        The simulated document, with optional metadata.

        Hide doc attributes Show doc attributes object
        • _id string Required
        • _index string Required
        • _ingest object Required
          Hide _ingest attributes Show _ingest attributes object
          • _redact object
            Hide _redact attribute Show _redact attribute object
            • _is_redacted boolean Required

              indicates if document has been redacted

          • timestamp string
          • pipeline string
        • _routing string

          Value used to send the document to a specific primary shard.

        • _source object Required

          JSON body for the document.

          Hide _source attribute Show _source attribute object
          • * object Additional properties
        • _version number | string

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

        • _version_type string

          Values are internal, external, external_gte, or force.

      • error object

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide error attributes Show error attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • processor_results array[object]
        Hide processor_results attributes Show processor_results attributes object
        • doc object

          The simulated document, with optional metadata.

          Hide doc attributes Show doc attributes object
          • _id string Required
          • _index string Required
          • _ingest object Required
          • _routing string

            Value used to send the document to a specific primary shard.

          • _source object Required

            JSON body for the document.

          • _version
          • _version_type string

            Values are internal, external, external_gte, or force.

        • tag string
        • processor_type string
        • status string

          Values are success, error, error_ignored, skipped, or dropped.

        • description string
        • ignored_error object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide ignored_error attributes Show ignored_error attributes object
          • type string Required

            The type of error

          • reason
          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]
          • suppressed array[object]
        • error object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide error attributes Show error attributes object
          • type string Required

            The type of error

          • reason
          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]
          • suppressed array[object]
POST /_ingest/pipeline/_simulate
curl \
 --request POST 'http://api.example.com/_ingest/pipeline/_simulate' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"pipeline\" :\n  {\n    \"description\": \"_description\",\n    \"processors\": [\n      {\n        \"set\" : {\n          \"field\" : \"field2\",\n          \"value\" : \"_value\"\n        }\n      }\n    ]\n  },\n  \"docs\": [\n    {\n      \"_index\": \"index\",\n      \"_id\": \"id\",\n      \"_source\": {\n        \"foo\": \"bar\"\n      }\n    },\n    {\n      \"_index\": \"index\",\n      \"_id\": \"id\",\n      \"_source\": {\n        \"foo\": \"rab\"\n      }\n    }\n  ]\n}"'
Request example
You can specify the used pipeline either in the request body or as a path parameter.
{
  "pipeline" :
  {
    "description": "_description",
    "processors": [
      {
        "set" : {
          "field" : "field2",
          "value" : "_value"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}
Response examples (200)
A successful response for running an ingest pipeline against a set of provided documents.
{
   "docs": [
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_version": "-3",
            "_source": {
               "field2": "_value",
               "foo": "bar"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.187Z"
            }
         }
      },
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_version": "-3",
            "_source": {
               "field2": "_value",
               "foo": "rab"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.188Z"
            }
         }
      }
   ]
}

Simulate a pipeline Generally available; Added in 5.0.0

GET /_ingest/pipeline/{id}/_simulate

Run an ingest pipeline against a set of provided documents. You can either specify an existing pipeline to use with the provided documents or supply a pipeline definition in the body of the request.

Required authorization

  • Cluster privileges: read_pipeline

Path parameters

  • id string Required

    The pipeline to test. If you don't specify a pipeline in the request body, this parameter is required.

Query parameters

  • verbose boolean

    If true, the response includes output data for each processor in the executed pipeline.

application/json

Body Required

  • docs array[object] Required

    Sample documents to test in the pipeline.

    Hide docs attributes Show docs attributes object
    • _id string
    • _index string
    • _source object Required

      JSON body for the document.

  • pipeline object Additional properties
    Hide pipeline attributes Show pipeline attributes object
    • description string

      Description of the ingest pipeline.

    • on_failure array[object]

      Processors to run immediately after a processor failure.

      Hide on_failure attributes Show on_failure attributes object
      • append object
        Hide append attributes Show append attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • value object | array[object] Required

          The value to be appended. Supports template snippets.

        • allow_duplicates boolean

          If false, the processor does not append values already present in the field.

      • attachment object
        Hide attachment attributes Show attachment attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • indexed_chars number

          The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

        • indexed_chars_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • remove_binary boolean

          If true, the binary field will be removed from the document

        • resource_name string

          Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

      • bytes object
        Hide bytes attributes Show bytes attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • circle object
        Hide circle attributes Show circle attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • error_distance number Required

          The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • shape_type string Required

          Values are geo_shape or shape.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • community_id object
        Hide community_id attributes Show community_id attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • source_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • source_port string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_port string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • iana_number string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • icmp_type string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • icmp_code string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • transport string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • seed number

          Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • convert object
        Hide convert attributes Show convert attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • type string Required

          Values are integer, long, double, float, boolean, ip, string, or auto.

      • csv object
        Hide csv attributes Show csv attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • empty_value object

          Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • quote string

          Quote used in CSV, has to be single character string.

        • separator string

          Separator used in CSV, has to be single character string.

        • target_fields string | array[string] Required
        • trim boolean

          Trim whitespaces in unquoted fields.

      • date object
        Hide date attributes Show date attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • formats array[string] Required

          An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

        • locale string

          The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • timezone string

          The timezone to use when parsing the date. Supports template snippets.

        • output_format string

          The format to use when writing the date to target_field. Must be a valid java time pattern.

      • date_index_name object
        Hide date_index_name attributes Show date_index_name attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • date_formats array[string] Required

          An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

        • date_rounding string Required

          How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • index_name_format string

          The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

        • index_name_prefix string

          A prefix of the index name to be prepended before the printed date. Supports template snippets.

        • locale string

          The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

        • timezone string

          The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

      • dissect object
        Hide dissect attributes Show dissect attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • append_separator string

          The character(s) that separate the appended fields.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern string Required

          The pattern to apply to the field.

      • dot_expander object
        Hide dot_expander attributes Show dot_expander attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • override boolean

          Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

        • path string

          The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

      • drop object
        Hide drop attributes Show drop attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

      • enrich object
        Hide enrich attributes Show enrich attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • max_matches number

          The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

        • override boolean

          If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        • policy_name string Required

          The name of the enrich policy to use.

        • shape_relation string

          Values are intersects, disjoint, within, or contains.

        • target_field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • fail object
        Hide fail attributes Show fail attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • message string Required

          The error message thrown by the processor. Supports template snippets.

      • fingerprint object
        Hide fingerprint attributes Show fingerprint attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • fields string | array[string] Required
        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • salt string

          Salt value for the hash function.

        • method string

          Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

        • ignore_missing boolean

          If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

      • foreach object
        Hide foreach attributes Show foreach attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true, the processor silently exits without changing the document if the field is null or missing.

        • processor object Required
      • ip_location object
        Hide ip_location attributes Show ip_location attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • database_file string

          The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • first_only boolean

          If true, only the first found IP location data will be returned, even if the field contains an array.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • properties array[string]

          Controls what properties are added to the target_field based on the IP location lookup.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • download_database_on_pipeline_creation boolean

          If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

      • geo_grid object
        Hide geo_grid attributes Show geo_grid attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          The field to interpret as a geo-tile.= The field format is determined by the tile_type.

        • tile_type string Required

          Values are geotile, geohex, or geohash.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • parent_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • children_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • non_children_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • precision_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_format string

          Values are geojson or wkt.

      • geoip object
        Hide geoip attributes Show geoip attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • database_file string

          The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • first_only boolean

          If true, only the first found geoip data will be returned, even if the field contains an array.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • properties array[string]

          Controls what properties are added to the target_field based on the geoip lookup.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • download_database_on_pipeline_creation boolean

          If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

      • grok object
        Hide grok attributes Show grok attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • ecs_compatibility string

          Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern_definitions object

          A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          Hide pattern_definitions attribute Show pattern_definitions attribute object
          • * string Additional properties
        • patterns array[string] Required

          An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

        • trace_match boolean

          When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

      • gsub object
        Hide gsub attributes Show gsub attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern string Required

          The pattern to be replaced.

        • replacement string Required

          The string to replace the matching patterns with.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • html_strip object
        Hide html_strip attributes Show html_strip attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document,

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • inference object
        Hide inference attributes Show inference attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • model_id string Required
        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field_map object

          Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          Hide field_map attribute Show field_map attribute object
          • * object Additional properties
        • inference_config object
          Hide inference_config attributes Show inference_config attributes object
          • regression object
          • classification object
        • input_output object | array[object]

          Input fields for inference and output (destination) fields for the inference results. This option is incompatible with the target_field and field_map options.

        • ignore_missing boolean

          If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

      • join object
        Hide join attributes Show join attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • separator string Required

          The separator character.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • json object
        Hide json attributes Show json attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • add_to_root boolean

          Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

        • add_to_root_conflict_strategy string

          Values are replace or merge.

        • allow_duplicate_keys boolean

          When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • kv object
        Hide kv attributes Show kv attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • exclude_keys array[string]

          List of keys to exclude from document.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field_split string Required

          Regex pattern to use for splitting key-value pairs.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • include_keys array[string]

          List of keys to filter and insert into document. Defaults to including all keys.

        • prefix string

          Prefix to be added to extracted keys.

        • strip_brackets boolean

          If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • trim_key string

          String of characters to trim from extracted keys.

        • trim_value string

          String of characters to trim from extracted values.

        • value_split string Required

          Regex pattern to use for splitting the key from the value within a key-value pair.

      • lowercase object
        Hide lowercase attributes Show lowercase attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • network_direction object
        Hide network_direction attributes Show network_direction attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • source_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • internal_networks array[string]

          List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

        • internal_networks_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • pipeline object
        Hide pipeline attributes Show pipeline attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • name string Required
        • ignore_missing_pipeline boolean

          Whether to ignore missing pipelines instead of failing.

      • redact object
        Hide redact attributes Show redact attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • patterns array[string] Required

          A list of grok expressions to match and redact named captures with

        • pattern_definitions object
          Hide pattern_definitions attribute Show pattern_definitions attribute object
          • * string Additional properties
        • prefix string

          Start a redacted section with this token

        • suffix string

          End a redacted section with this token

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • skip_if_unlicensed boolean

          If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

        • trace_redact boolean Generally available; Added in 8.16.0

          If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

      • registered_domain object
        Hide registered_domain attributes Show registered_domain attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • remove object
        Hide remove attributes Show remove attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string | array[string] Required
        • keep string | array[string]
        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • rename object
        Hide rename attributes Show rename attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • reroute object
        Hide reroute attributes Show reroute attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • destination string

          A static value for the target. Can’t be set when the dataset or namespace option is set.

        • dataset string | array[string]

          Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.

          Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

          default {{data_stream.dataset}}

        • namespace string | array[string]

          Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.

          Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

          default {{data_stream.namespace}}

      • script object
        Hide script attributes Show script attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • id string
        • lang string

          Script language.

        • params object

          Object containing parameters for the script.

          Hide params attribute Show params attribute object
          • * object Additional properties
        • source string

          Inline script. If no id is specified, this parameter is required.

      • set object
        Hide set attributes Show set attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • copy_from string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_empty_value boolean

          If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

        • media_type string

          The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

        • override boolean

          If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        • value object

          The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

      • set_security_user object
        Hide set_security_user attributes Show set_security_user attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Controls what user related properties are added to the field.

      • sort object
        Hide sort attributes Show sort attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • order string

          Values are asc or desc.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • split object
        Hide split attributes Show split attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • preserve_trailing boolean

          Preserves empty trailing fields, if any.

        • separator string Required

          A regex which matches the separator, for example, , or \s+.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • terminate object
        Hide terminate attributes Show terminate attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

      • trim object
        Hide trim attributes Show trim attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • uppercase object
        Hide uppercase attributes Show uppercase attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • urldecode object
        Hide urldecode attributes Show urldecode attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • uri_parts object
        Hide uri_parts attributes Show uri_parts attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • keep_original boolean

          If true, the processor copies the unparsed URI to <target_field>.original.

        • remove_if_successful boolean

          If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • user_agent object
        Hide user_agent attributes Show user_agent attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • regex_file string

          The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Controls what properties are added to target_field.

          Values are name, os, device, original, or version.

        • extract_device_type boolean Beta; Added in 8.9.0

          Extracts device type from the user agent string on a best-effort basis.

    • processors array[object]

      Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.

      Hide processors attributes Show processors attributes object
      • append object
        Hide append attributes Show append attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • value object | array[object] Required

          The value to be appended. Supports template snippets.

        • allow_duplicates boolean

          If false, the processor does not append values already present in the field.

      • attachment object
        Hide attachment attributes Show attachment attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • indexed_chars number

          The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

        • indexed_chars_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • remove_binary boolean

          If true, the binary field will be removed from the document

        • resource_name string

          Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

      • bytes object
        Hide bytes attributes Show bytes attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • circle object
        Hide circle attributes Show circle attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • error_distance number Required

          The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • shape_type string Required

          Values are geo_shape or shape.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • community_id object
        Hide community_id attributes Show community_id attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • source_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • source_port string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_port string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • iana_number string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • icmp_type string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • icmp_code string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • transport string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • seed number

          Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • convert object
        Hide convert attributes Show convert attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • type string Required

          Values are integer, long, double, float, boolean, ip, string, or auto.

      • csv object
        Hide csv attributes Show csv attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • empty_value object

          Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • quote string

          Quote used in CSV, has to be single character string.

        • separator string

          Separator used in CSV, has to be single character string.

        • target_fields string | array[string] Required
        • trim boolean

          Trim whitespaces in unquoted fields.

      • date object
        Hide date attributes Show date attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • formats array[string] Required

          An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

        • locale string

          The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • timezone string

          The timezone to use when parsing the date. Supports template snippets.

        • output_format string

          The format to use when writing the date to target_field. Must be a valid java time pattern.

      • date_index_name object
        Hide date_index_name attributes Show date_index_name attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • date_formats array[string] Required

          An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

        • date_rounding string Required

          How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • index_name_format string

          The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

        • index_name_prefix string

          A prefix of the index name to be prepended before the printed date. Supports template snippets.

        • locale string

          The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

        • timezone string

          The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

      • dissect object
        Hide dissect attributes Show dissect attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • append_separator string

          The character(s) that separate the appended fields.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern string Required

          The pattern to apply to the field.

      • dot_expander object
        Hide dot_expander attributes Show dot_expander attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • override boolean

          Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

        • path string

          The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

      • drop object
        Hide drop attributes Show drop attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

      • enrich object
        Hide enrich attributes Show enrich attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • max_matches number

          The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

        • override boolean

          If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        • policy_name string Required

          The name of the enrich policy to use.

        • shape_relation string

          Values are intersects, disjoint, within, or contains.

        • target_field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • fail object
        Hide fail attributes Show fail attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • message string Required

          The error message thrown by the processor. Supports template snippets.

      • fingerprint object
        Hide fingerprint attributes Show fingerprint attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • fields string | array[string] Required
        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • salt string

          Salt value for the hash function.

        • method string

          Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

        • ignore_missing boolean

          If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

      • foreach object
        Hide foreach attributes Show foreach attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true, the processor silently exits without changing the document if the field is null or missing.

        • processor object Required
      • ip_location object
        Hide ip_location attributes Show ip_location attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • database_file string

          The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • first_only boolean

          If true, only the first found IP location data will be returned, even if the field contains an array.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • properties array[string]

          Controls what properties are added to the target_field based on the IP location lookup.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • download_database_on_pipeline_creation boolean

          If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

      • geo_grid object
        Hide geo_grid attributes Show geo_grid attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          The field to interpret as a geo-tile.= The field format is determined by the tile_type.

        • tile_type string Required

          Values are geotile, geohex, or geohash.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • parent_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • children_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • non_children_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • precision_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_format string

          Values are geojson or wkt.

      • geoip object
        Hide geoip attributes Show geoip attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • database_file string

          The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • first_only boolean

          If true, only the first found geoip data will be returned, even if the field contains an array.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • properties array[string]

          Controls what properties are added to the target_field based on the geoip lookup.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • download_database_on_pipeline_creation boolean

          If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

      • grok object
        Hide grok attributes Show grok attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • ecs_compatibility string

          Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern_definitions object

          A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          Hide pattern_definitions attribute Show pattern_definitions attribute object
          • * string Additional properties
        • patterns array[string] Required

          An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

        • trace_match boolean

          When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

      • gsub object
        Hide gsub attributes Show gsub attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern string Required

          The pattern to be replaced.

        • replacement string Required

          The string to replace the matching patterns with.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • html_strip object
        Hide html_strip attributes Show html_strip attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document,

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • inference object
        Hide inference attributes Show inference attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • model_id string Required
        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field_map object

          Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          Hide field_map attribute Show field_map attribute object
          • * object Additional properties
        • inference_config object
          Hide inference_config attributes Show inference_config attributes object
          • regression object
          • classification object
        • input_output object | array[object]

          Input fields for inference and output (destination) fields for the inference results. This option is incompatible with the target_field and field_map options.

        • ignore_missing boolean

          If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

      • join object
        Hide join attributes Show join attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • separator string Required

          The separator character.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • json object
        Hide json attributes Show json attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • add_to_root boolean

          Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

        • add_to_root_conflict_strategy string

          Values are replace or merge.

        • allow_duplicate_keys boolean

          When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • kv object
        Hide kv attributes Show kv attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • exclude_keys array[string]

          List of keys to exclude from document.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field_split string Required

          Regex pattern to use for splitting key-value pairs.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • include_keys array[string]

          List of keys to filter and insert into document. Defaults to including all keys.

        • prefix string

          Prefix to be added to extracted keys.

        • strip_brackets boolean

          If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • trim_key string

          String of characters to trim from extracted keys.

        • trim_value string

          String of characters to trim from extracted values.

        • value_split string Required

          Regex pattern to use for splitting the key from the value within a key-value pair.

      • lowercase object
        Hide lowercase attributes Show lowercase attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • network_direction object
        Hide network_direction attributes Show network_direction attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • source_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • internal_networks array[string]

          List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

        • internal_networks_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • pipeline object
        Hide pipeline attributes Show pipeline attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • name string Required
        • ignore_missing_pipeline boolean

          Whether to ignore missing pipelines instead of failing.

      • redact object
        Hide redact attributes Show redact attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • patterns array[string] Required

          A list of grok expressions to match and redact named captures with

        • pattern_definitions object
          Hide pattern_definitions attribute Show pattern_definitions attribute object
          • * string Additional properties
        • prefix string

          Start a redacted section with this token

        • suffix string

          End a redacted section with this token

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • skip_if_unlicensed boolean

          If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

        • trace_redact boolean Generally available; Added in 8.16.0

          If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

      • registered_domain object
        Hide registered_domain attributes Show registered_domain attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • remove object
        Hide remove attributes Show remove attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string | array[string] Required
        • keep string | array[string]
        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • rename object
        Hide rename attributes Show rename attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • reroute object
        Hide reroute attributes Show reroute attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • destination string

          A static value for the target. Can’t be set when the dataset or namespace option is set.

        • dataset string | array[string]

          Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.

          Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

          default {{data_stream.dataset}}

        • namespace string | array[string]

          Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.

          Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

          default {{data_stream.namespace}}

      • script object
        Hide script attributes Show script attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • id string
        • lang string

          Script language.

        • params object

          Object containing parameters for the script.

          Hide params attribute Show params attribute object
          • * object Additional properties
        • source string

          Inline script. If no id is specified, this parameter is required.

      • set object
        Hide set attributes Show set attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • copy_from string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_empty_value boolean

          If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

        • media_type string

          The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

        • override boolean

          If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        • value object

          The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

      • set_security_user object
        Hide set_security_user attributes Show set_security_user attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Controls what user related properties are added to the field.

      • sort object
        Hide sort attributes Show sort attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • order string

          Values are asc or desc.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • split object
        Hide split attributes Show split attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • preserve_trailing boolean

          Preserves empty trailing fields, if any.

        • separator string Required

          A regex which matches the separator, for example, , or \s+.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • terminate object
        Hide terminate attributes Show terminate attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

      • trim object
        Hide trim attributes Show trim attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • uppercase object
        Hide uppercase attributes Show uppercase attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • urldecode object
        Hide urldecode attributes Show urldecode attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • uri_parts object
        Hide uri_parts attributes Show uri_parts attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • keep_original boolean

          If true, the processor copies the unparsed URI to <target_field>.original.

        • remove_if_successful boolean

          If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • user_agent object
        Hide user_agent attributes Show user_agent attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • regex_file string

          The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Controls what properties are added to target_field.

          Values are name, os, device, original, or version.

        • extract_device_type boolean Beta; Added in 8.9.0

          Extracts device type from the user agent string on a best-effort basis.

    • version number
    • deprecated boolean

      Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.

    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required
      Hide docs attributes Show docs attributes object
      • doc object

        The simulated document, with optional metadata.

        Hide doc attributes Show doc attributes object
        • _id string Required
        • _index string Required
        • _ingest object Required
          Hide _ingest attributes Show _ingest attributes object
          • _redact object
            Hide _redact attribute Show _redact attribute object
            • _is_redacted boolean Required

              indicates if document has been redacted

          • timestamp string
          • pipeline string
        • _routing string

          Value used to send the document to a specific primary shard.

        • _source object Required

          JSON body for the document.

          Hide _source attribute Show _source attribute object
          • * object Additional properties
        • _version number | string

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

        • _version_type string

          Values are internal, external, external_gte, or force.

      • error object

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide error attributes Show error attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • processor_results array[object]
        Hide processor_results attributes Show processor_results attributes object
        • doc object

          The simulated document, with optional metadata.

          Hide doc attributes Show doc attributes object
          • _id string Required
          • _index string Required
          • _ingest object Required
          • _routing string

            Value used to send the document to a specific primary shard.

          • _source object Required

            JSON body for the document.

          • _version
          • _version_type string

            Values are internal, external, external_gte, or force.

        • tag string
        • processor_type string
        • status string

          Values are success, error, error_ignored, skipped, or dropped.

        • description string
        • ignored_error object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide ignored_error attributes Show ignored_error attributes object
          • type string Required

            The type of error

          • reason
          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]
          • suppressed array[object]
        • error object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide error attributes Show error attributes object
          • type string Required

            The type of error

          • reason
          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]
          • suppressed array[object]
GET /_ingest/pipeline/{id}/_simulate
curl \
 --request GET 'http://api.example.com/_ingest/pipeline/{id}/_simulate' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"pipeline\" :\n  {\n    \"description\": \"_description\",\n    \"processors\": [\n      {\n        \"set\" : {\n          \"field\" : \"field2\",\n          \"value\" : \"_value\"\n        }\n      }\n    ]\n  },\n  \"docs\": [\n    {\n      \"_index\": \"index\",\n      \"_id\": \"id\",\n      \"_source\": {\n        \"foo\": \"bar\"\n      }\n    },\n    {\n      \"_index\": \"index\",\n      \"_id\": \"id\",\n      \"_source\": {\n        \"foo\": \"rab\"\n      }\n    }\n  ]\n}"'
Request example
You can specify the used pipeline either in the request body or as a path parameter.
{
  "pipeline" :
  {
    "description": "_description",
    "processors": [
      {
        "set" : {
          "field" : "field2",
          "value" : "_value"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}
Response examples (200)
A successful response for running an ingest pipeline against a set of provided documents.
{
   "docs": [
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_version": "-3",
            "_source": {
               "field2": "_value",
               "foo": "bar"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.187Z"
            }
         }
      },
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_version": "-3",
            "_source": {
               "field2": "_value",
               "foo": "rab"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.188Z"
            }
         }
      }
   ]
}

Simulate a pipeline Generally available; Added in 5.0.0

POST /_ingest/pipeline/{id}/_simulate

Run an ingest pipeline against a set of provided documents. You can either specify an existing pipeline to use with the provided documents or supply a pipeline definition in the body of the request.

Required authorization

  • Cluster privileges: read_pipeline

Path parameters

  • id string Required

    The pipeline to test. If you don't specify a pipeline in the request body, this parameter is required.

Query parameters

  • verbose boolean

    If true, the response includes output data for each processor in the executed pipeline.

application/json

Body Required

  • docs array[object] Required

    Sample documents to test in the pipeline.

    Hide docs attributes Show docs attributes object
    • _id string
    • _index string
    • _source object Required

      JSON body for the document.

  • pipeline object Additional properties
    Hide pipeline attributes Show pipeline attributes object
    • description string

      Description of the ingest pipeline.

    • on_failure array[object]

      Processors to run immediately after a processor failure.

      Hide on_failure attributes Show on_failure attributes object
      • append object
        Hide append attributes Show append attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • value object | array[object] Required

          The value to be appended. Supports template snippets.

        • allow_duplicates boolean

          If false, the processor does not append values already present in the field.

      • attachment object
        Hide attachment attributes Show attachment attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • indexed_chars number

          The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

        • indexed_chars_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • remove_binary boolean

          If true, the binary field will be removed from the document

        • resource_name string

          Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

      • bytes object
        Hide bytes attributes Show bytes attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • circle object
        Hide circle attributes Show circle attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • error_distance number Required

          The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • shape_type string Required

          Values are geo_shape or shape.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • community_id object
        Hide community_id attributes Show community_id attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • source_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • source_port string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_port string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • iana_number string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • icmp_type string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • icmp_code string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • transport string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • seed number

          Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • convert object
        Hide convert attributes Show convert attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • type string Required

          Values are integer, long, double, float, boolean, ip, string, or auto.

      • csv object
        Hide csv attributes Show csv attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • empty_value object

          Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • quote string

          Quote used in CSV, has to be single character string.

        • separator string

          Separator used in CSV, has to be single character string.

        • target_fields string | array[string] Required
        • trim boolean

          Trim whitespaces in unquoted fields.

      • date object
        Hide date attributes Show date attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • formats array[string] Required

          An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

        • locale string

          The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • timezone string

          The timezone to use when parsing the date. Supports template snippets.

        • output_format string

          The format to use when writing the date to target_field. Must be a valid java time pattern.

      • date_index_name object
        Hide date_index_name attributes Show date_index_name attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • date_formats array[string] Required

          An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

        • date_rounding string Required

          How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • index_name_format string

          The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

        • index_name_prefix string

          A prefix of the index name to be prepended before the printed date. Supports template snippets.

        • locale string

          The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

        • timezone string

          The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

      • dissect object
        Hide dissect attributes Show dissect attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • append_separator string

          The character(s) that separate the appended fields.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern string Required

          The pattern to apply to the field.

      • dot_expander object
        Hide dot_expander attributes Show dot_expander attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • override boolean

          Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

        • path string

          The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

      • drop object
        Hide drop attributes Show drop attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

      • enrich object
        Hide enrich attributes Show enrich attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • max_matches number

          The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

        • override boolean

          If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        • policy_name string Required

          The name of the enrich policy to use.

        • shape_relation string

          Values are intersects, disjoint, within, or contains.

        • target_field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • fail object
        Hide fail attributes Show fail attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • message string Required

          The error message thrown by the processor. Supports template snippets.

      • fingerprint object
        Hide fingerprint attributes Show fingerprint attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • fields string | array[string] Required
        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • salt string

          Salt value for the hash function.

        • method string

          Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

        • ignore_missing boolean

          If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

      • foreach object
        Hide foreach attributes Show foreach attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true, the processor silently exits without changing the document if the field is null or missing.

        • processor object Required
      • ip_location object
        Hide ip_location attributes Show ip_location attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • database_file string

          The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • first_only boolean

          If true, only the first found IP location data will be returned, even if the field contains an array.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • properties array[string]

          Controls what properties are added to the target_field based on the IP location lookup.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • download_database_on_pipeline_creation boolean

          If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

      • geo_grid object
        Hide geo_grid attributes Show geo_grid attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          The field to interpret as a geo-tile.= The field format is determined by the tile_type.

        • tile_type string Required

          Values are geotile, geohex, or geohash.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • parent_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • children_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • non_children_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • precision_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_format string

          Values are geojson or wkt.

      • geoip object
        Hide geoip attributes Show geoip attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • database_file string

          The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • first_only boolean

          If true, only the first found geoip data will be returned, even if the field contains an array.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • properties array[string]

          Controls what properties are added to the target_field based on the geoip lookup.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • download_database_on_pipeline_creation boolean

          If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

      • grok object
        Hide grok attributes Show grok attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • ecs_compatibility string

          Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern_definitions object

          A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          Hide pattern_definitions attribute Show pattern_definitions attribute object
          • * string Additional properties
        • patterns array[string] Required

          An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

        • trace_match boolean

          When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

      • gsub object
        Hide gsub attributes Show gsub attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern string Required

          The pattern to be replaced.

        • replacement string Required

          The string to replace the matching patterns with.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • html_strip object
        Hide html_strip attributes Show html_strip attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document,

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • inference object
        Hide inference attributes Show inference attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • model_id string Required
        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field_map object

          Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          Hide field_map attribute Show field_map attribute object
          • * object Additional properties
        • inference_config object
          Hide inference_config attributes Show inference_config attributes object
          • regression object
          • classification object
        • input_output object | array[object]

          Input fields for inference and output (destination) fields for the inference results. This option is incompatible with the target_field and field_map options.

        • ignore_missing boolean

          If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

      • join object
        Hide join attributes Show join attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • separator string Required

          The separator character.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • json object
        Hide json attributes Show json attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • add_to_root boolean

          Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

        • add_to_root_conflict_strategy string

          Values are replace or merge.

        • allow_duplicate_keys boolean

          When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • kv object
        Hide kv attributes Show kv attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • exclude_keys array[string]

          List of keys to exclude from document.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field_split string Required

          Regex pattern to use for splitting key-value pairs.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • include_keys array[string]

          List of keys to filter and insert into document. Defaults to including all keys.

        • prefix string

          Prefix to be added to extracted keys.

        • strip_brackets boolean

          If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • trim_key string

          String of characters to trim from extracted keys.

        • trim_value string

          String of characters to trim from extracted values.

        • value_split string Required

          Regex pattern to use for splitting the key from the value within a key-value pair.

      • lowercase object
        Hide lowercase attributes Show lowercase attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • network_direction object
        Hide network_direction attributes Show network_direction attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • source_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • internal_networks array[string]

          List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

        • internal_networks_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • pipeline object
        Hide pipeline attributes Show pipeline attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • name string Required
        • ignore_missing_pipeline boolean

          Whether to ignore missing pipelines instead of failing.

      • redact object
        Hide redact attributes Show redact attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • patterns array[string] Required

          A list of grok expressions to match and redact named captures with

        • pattern_definitions object
          Hide pattern_definitions attribute Show pattern_definitions attribute object
          • * string Additional properties
        • prefix string

          Start a redacted section with this token

        • suffix string

          End a redacted section with this token

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • skip_if_unlicensed boolean

          If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

        • trace_redact boolean Generally available; Added in 8.16.0

          If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

      • registered_domain object
        Hide registered_domain attributes Show registered_domain attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • remove object
        Hide remove attributes Show remove attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string | array[string] Required
        • keep string | array[string]
        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • rename object
        Hide rename attributes Show rename attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • reroute object
        Hide reroute attributes Show reroute attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • destination string

          A static value for the target. Can’t be set when the dataset or namespace option is set.

        • dataset string | array[string]

          Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.

          Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

          default {{data_stream.dataset}}

        • namespace string | array[string]

          Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.

          Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

          default {{data_stream.namespace}}

      • script object
        Hide script attributes Show script attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • id string
        • lang string

          Script language.

        • params object

          Object containing parameters for the script.

          Hide params attribute Show params attribute object
          • * object Additional properties
        • source string

          Inline script. If no id is specified, this parameter is required.

      • set object
        Hide set attributes Show set attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • copy_from string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_empty_value boolean

          If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

        • media_type string

          The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

        • override boolean

          If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        • value object

          The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

      • set_security_user object
        Hide set_security_user attributes Show set_security_user attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Controls what user related properties are added to the field.

      • sort object
        Hide sort attributes Show sort attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • order string

          Values are asc or desc.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • split object
        Hide split attributes Show split attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • preserve_trailing boolean

          Preserves empty trailing fields, if any.

        • separator string Required

          A regex which matches the separator, for example, , or \s+.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • terminate object
        Hide terminate attributes Show terminate attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

      • trim object
        Hide trim attributes Show trim attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • uppercase object
        Hide uppercase attributes Show uppercase attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • urldecode object
        Hide urldecode attributes Show urldecode attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • uri_parts object
        Hide uri_parts attributes Show uri_parts attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • keep_original boolean

          If true, the processor copies the unparsed URI to <target_field>.original.

        • remove_if_successful boolean

          If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • user_agent object
        Hide user_agent attributes Show user_agent attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • regex_file string

          The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Controls what properties are added to target_field.

          Values are name, os, device, original, or version.

        • extract_device_type boolean Beta; Added in 8.9.0

          Extracts device type from the user agent string on a best-effort basis.

    • processors array[object]

      Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.

      Hide processors attributes Show processors attributes object
      • append object
        Hide append attributes Show append attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • value object | array[object] Required

          The value to be appended. Supports template snippets.

        • allow_duplicates boolean

          If false, the processor does not append values already present in the field.

      • attachment object
        Hide attachment attributes Show attachment attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • indexed_chars number

          The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

        • indexed_chars_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • remove_binary boolean

          If true, the binary field will be removed from the document

        • resource_name string

          Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

      • bytes object
        Hide bytes attributes Show bytes attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • circle object
        Hide circle attributes Show circle attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • error_distance number Required

          The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • shape_type string Required

          Values are geo_shape or shape.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • community_id object
        Hide community_id attributes Show community_id attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • source_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • source_port string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_port string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • iana_number string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • icmp_type string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • icmp_code string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • transport string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • seed number

          Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • convert object
        Hide convert attributes Show convert attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • type string Required

          Values are integer, long, double, float, boolean, ip, string, or auto.

      • csv object
        Hide csv attributes Show csv attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • empty_value object

          Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • quote string

          Quote used in CSV, has to be single character string.

        • separator string

          Separator used in CSV, has to be single character string.

        • target_fields string | array[string] Required
        • trim boolean

          Trim whitespaces in unquoted fields.

      • date object
        Hide date attributes Show date attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • formats array[string] Required

          An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

        • locale string

          The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • timezone string

          The timezone to use when parsing the date. Supports template snippets.

        • output_format string

          The format to use when writing the date to target_field. Must be a valid java time pattern.

      • date_index_name object
        Hide date_index_name attributes Show date_index_name attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • date_formats array[string] Required

          An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

        • date_rounding string Required

          How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • index_name_format string

          The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

        • index_name_prefix string

          A prefix of the index name to be prepended before the printed date. Supports template snippets.

        • locale string

          The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

        • timezone string

          The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

      • dissect object
        Hide dissect attributes Show dissect attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • append_separator string

          The character(s) that separate the appended fields.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern string Required

          The pattern to apply to the field.

      • dot_expander object
        Hide dot_expander attributes Show dot_expander attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • override boolean

          Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

        • path string

          The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

      • drop object
        Hide drop attributes Show drop attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

      • enrich object
        Hide enrich attributes Show enrich attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • max_matches number

          The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

        • override boolean

          If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        • policy_name string Required

          The name of the enrich policy to use.

        • shape_relation string

          Values are intersects, disjoint, within, or contains.

        • target_field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • fail object
        Hide fail attributes Show fail attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • message string Required

          The error message thrown by the processor. Supports template snippets.

      • fingerprint object
        Hide fingerprint attributes Show fingerprint attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • fields string | array[string] Required
        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • salt string

          Salt value for the hash function.

        • method string

          Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

        • ignore_missing boolean

          If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

      • foreach object
        Hide foreach attributes Show foreach attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true, the processor silently exits without changing the document if the field is null or missing.

        • processor object Required
      • ip_location object
        Hide ip_location attributes Show ip_location attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • database_file string

          The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • first_only boolean

          If true, only the first found IP location data will be returned, even if the field contains an array.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • properties array[string]

          Controls what properties are added to the target_field based on the IP location lookup.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • download_database_on_pipeline_creation boolean

          If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

      • geo_grid object
        Hide geo_grid attributes Show geo_grid attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          The field to interpret as a geo-tile.= The field format is determined by the tile_type.

        • tile_type string Required

          Values are geotile, geohex, or geohash.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • parent_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • children_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • non_children_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • precision_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_format string

          Values are geojson or wkt.

      • geoip object
        Hide geoip attributes Show geoip attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • database_file string

          The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • first_only boolean

          If true, only the first found geoip data will be returned, even if the field contains an array.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • properties array[string]

          Controls what properties are added to the target_field based on the geoip lookup.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • download_database_on_pipeline_creation boolean

          If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

      • grok object
        Hide grok attributes Show grok attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • ecs_compatibility string

          Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern_definitions object

          A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          Hide pattern_definitions attribute Show pattern_definitions attribute object
          • * string Additional properties
        • patterns array[string] Required

          An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

        • trace_match boolean

          When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

      • gsub object
        Hide gsub attributes Show gsub attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern string Required

          The pattern to be replaced.

        • replacement string Required

          The string to replace the matching patterns with.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • html_strip object
        Hide html_strip attributes Show html_strip attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document,

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • inference object
        Hide inference attributes Show inference attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • model_id string Required
        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field_map object

          Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          Hide field_map attribute Show field_map attribute object
          • * object Additional properties
        • inference_config object
          Hide inference_config attributes Show inference_config attributes object
          • regression object
          • classification object
        • input_output object | array[object]

          Input fields for inference and output (destination) fields for the inference results. This option is incompatible with the target_field and field_map options.

        • ignore_missing boolean

          If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

      • join object
        Hide join attributes Show join attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • separator string Required

          The separator character.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • json object
        Hide json attributes Show json attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • add_to_root boolean

          Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

        • add_to_root_conflict_strategy string

          Values are replace or merge.

        • allow_duplicate_keys boolean

          When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • kv object
        Hide kv attributes Show kv attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • exclude_keys array[string]

          List of keys to exclude from document.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field_split string Required

          Regex pattern to use for splitting key-value pairs.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • include_keys array[string]

          List of keys to filter and insert into document. Defaults to including all keys.

        • prefix string

          Prefix to be added to extracted keys.

        • strip_brackets boolean

          If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • trim_key string

          String of characters to trim from extracted keys.

        • trim_value string

          String of characters to trim from extracted values.

        • value_split string Required

          Regex pattern to use for splitting the key from the value within a key-value pair.

      • lowercase object
        Hide lowercase attributes Show lowercase attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • network_direction object
        Hide network_direction attributes Show network_direction attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • source_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • destination_ip string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • internal_networks array[string]

          List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

        • internal_networks_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • pipeline object
        Hide pipeline attributes Show pipeline attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • name string Required
        • ignore_missing_pipeline boolean

          Whether to ignore missing pipelines instead of failing.

      • redact object
        Hide redact attributes Show redact attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • patterns array[string] Required

          A list of grok expressions to match and redact named captures with

        • pattern_definitions object
          Hide pattern_definitions attribute Show pattern_definitions attribute object
          • * string Additional properties
        • prefix string

          Start a redacted section with this token

        • suffix string

          End a redacted section with this token

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • skip_if_unlicensed boolean

          If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

        • trace_redact boolean Generally available; Added in 8.16.0

          If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

      • registered_domain object
        Hide registered_domain attributes Show registered_domain attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and any required fields are missing, the processor quietly exits without modifying the document.

      • remove object
        Hide remove attributes Show remove attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string | array[string] Required
        • keep string | array[string]
        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • rename object
        Hide rename attributes Show rename attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • reroute object
        Hide reroute attributes Show reroute attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • destination string

          A static value for the target. Can’t be set when the dataset or namespace option is set.

        • dataset string | array[string]

          Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.

          Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

          default {{data_stream.dataset}}

        • namespace string | array[string]

          Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.

          Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

          default {{data_stream.namespace}}

      • script object
        Hide script attributes Show script attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • id string
        • lang string

          Script language.

        • params object

          Object containing parameters for the script.

          Hide params attribute Show params attribute object
          • * object Additional properties
        • source string

          Inline script. If no id is specified, this parameter is required.

      • set object
        Hide set attributes Show set attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • copy_from string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_empty_value boolean

          If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

        • media_type string

          The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

        • override boolean

          If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        • value object

          The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

      • set_security_user object
        Hide set_security_user attributes Show set_security_user attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Controls what user related properties are added to the field.

      • sort object
        Hide sort attributes Show sort attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • order string

          Values are asc or desc.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • split object
        Hide split attributes Show split attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • preserve_trailing boolean

          Preserves empty trailing fields, if any.

        • separator string Required

          A regex which matches the separator, for example, , or \s+.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • terminate object
        Hide terminate attributes Show terminate attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

      • trim object
        Hide trim attributes Show trim attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • uppercase object
        Hide uppercase attributes Show uppercase attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • urldecode object
        Hide urldecode attributes Show urldecode attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • uri_parts object
        Hide uri_parts attributes Show uri_parts attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • keep_original boolean

          If true, the processor copies the unparsed URI to <target_field>.original.

        • remove_if_successful boolean

          If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • user_agent object
        Hide user_agent attributes Show user_agent attributes object
        • description string

          Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if string

          Conditionally execute the processor.

        • ignore_failure boolean

          Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • ignore_missing boolean

          If true and field does not exist, the processor quietly exits without modifying the document.

        • regex_file string

          The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Controls what properties are added to target_field.

          Values are name, os, device, original, or version.

        • extract_device_type boolean Beta; Added in 8.9.0

          Extracts device type from the user agent string on a best-effort basis.

    • version number
    • deprecated boolean

      Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.

    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required
      Hide docs attributes Show docs attributes object
      • doc object

        The simulated document, with optional metadata.

        Hide doc attributes Show doc attributes object
        • _id string Required
        • _index string Required
        • _ingest object Required
          Hide _ingest attributes Show _ingest attributes object
          • _redact object
            Hide _redact attribute Show _redact attribute object
            • _is_redacted boolean Required

              indicates if document has been redacted

          • timestamp string
          • pipeline string
        • _routing string

          Value used to send the document to a specific primary shard.

        • _source object Required

          JSON body for the document.

          Hide _source attribute Show _source attribute object
          • * object Additional properties
        • _version number | string

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

        • _version_type string

          Values are internal, external, external_gte, or force.

      • error object

        Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        Hide error attributes Show error attributes object
        • type string Required

          The type of error

        • reason string | null

          A human-readable explanation of the error, in English.

        • stack_trace string

          The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • caused_by object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • root_cause array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

        • suppressed array[object]

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

      • processor_results array[object]
        Hide processor_results attributes Show processor_results attributes object
        • doc object

          The simulated document, with optional metadata.

          Hide doc attributes Show doc attributes object
          • _id string Required
          • _index string Required
          • _ingest object Required
          • _routing string

            Value used to send the document to a specific primary shard.

          • _source object Required

            JSON body for the document.

          • _version
          • _version_type string

            Values are internal, external, external_gte, or force.

        • tag string
        • processor_type string
        • status string

          Values are success, error, error_ignored, skipped, or dropped.

        • description string
        • ignored_error object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide ignored_error attributes Show ignored_error attributes object
          • type string Required

            The type of error

          • reason
          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]
          • suppressed array[object]
        • error object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide error attributes Show error attributes object
          • type string Required

            The type of error

          • reason
          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]
          • suppressed array[object]
POST /_ingest/pipeline/{id}/_simulate
curl \
 --request POST 'http://api.example.com/_ingest/pipeline/{id}/_simulate' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"pipeline\" :\n  {\n    \"description\": \"_description\",\n    \"processors\": [\n      {\n        \"set\" : {\n          \"field\" : \"field2\",\n          \"value\" : \"_value\"\n        }\n      }\n    ]\n  },\n  \"docs\": [\n    {\n      \"_index\": \"index\",\n      \"_id\": \"id\",\n      \"_source\": {\n        \"foo\": \"bar\"\n      }\n    },\n    {\n      \"_index\": \"index\",\n      \"_id\": \"id\",\n      \"_source\": {\n        \"foo\": \"rab\"\n      }\n    }\n  ]\n}"'
Request example
You can specify the used pipeline either in the request body or as a path parameter.
{
  "pipeline" :
  {
    "description": "_description",
    "processors": [
      {
        "set" : {
          "field" : "field2",
          "value" : "_value"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}
Response examples (200)
A successful response for running an ingest pipeline against a set of provided documents.
{
   "docs": [
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_version": "-3",
            "_source": {
               "field2": "_value",
               "foo": "bar"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.187Z"
            }
         }
      },
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_version": "-3",
            "_source": {
               "field2": "_value",
               "foo": "rab"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.188Z"
            }
         }
      }
   ]
}








Simulate data ingestion Technical preview; Added in 8.12.0

GET /_ingest/{index}/_simulate

Run ingest pipelines against a set of provided documents, optionally with substitute pipeline definitions, to simulate ingesting data into an index.

This API is meant to be used for troubleshooting or pipeline development, as it does not actually index any data into Elasticsearch.

The API runs the default and final pipeline for that index against a set of documents provided in the body of the request. If a pipeline contains a reroute processor, it follows that reroute processor to the new index, running that index's pipelines as well the same way that a non-simulated ingest would. No data is indexed into Elasticsearch. Instead, the transformed document is returned, along with the list of pipelines that have been run and the name of the index where the document would have been indexed if this were not a simulation. The transformed document is validated against the mappings that would apply to this index, and any validation error is reported in the result.

This API differs from the simulate pipeline API in that you specify a single pipeline for that API, and it runs only that one pipeline. The simulate pipeline API is more useful for developing a single pipeline, while the simulate ingest API is more useful for troubleshooting the interaction of the various pipelines that get applied when ingesting into an index.

By default, the pipeline definitions that are currently in the system are used. However, you can supply substitute pipeline definitions in the body of the request. These will be used in place of the pipeline definitions that are already in the system. This can be used to replace existing pipeline definitions or to create new ones. The pipeline substitutions are used only within this request.

Required authorization

  • Index privileges: index

Path parameters

  • index string Required

    The index to simulate ingesting into. This value can be overridden by specifying an index on each document. If you specify this parameter in the request path, it is used for any documents that do not explicitly specify an index argument.

Query parameters

  • pipeline string

    The pipeline to use as the default pipeline. This value can be used to override the default pipeline of the index.

application/json

Body Required

  • docs array[object] Required

    Sample documents to test in the pipeline.

    Hide docs attributes Show docs attributes object
    • _id string
    • _index string
    • _source object Required

      JSON body for the document.

  • component_template_substitutions object

    A map of component template names to substitute component template definition objects.

    Hide component_template_substitutions attribute Show component_template_substitutions attribute object
    • * object
      Hide * attributes Show * attributes object
      • template object Required
        Hide template attributes Show template attributes object
        • _meta object
          Hide _meta attribute Show _meta attribute object
          • * object Additional properties
        • version number
        • settings object
          Hide settings attribute Show settings attribute object
        • mappings object
          Hide mappings attributes Show mappings attributes object
          • all_field object
            Hide all_field attributes Show all_field attributes object
            • analyzer string Required
            • enabled boolean Required
            • omit_norms boolean Required
            • search_analyzer string Required
            • similarity string Required
            • store boolean Required
            • store_term_vector_offsets boolean Required
            • store_term_vector_payloads boolean Required
            • store_term_vector_positions boolean Required
            • store_term_vectors boolean Required
          • date_detection boolean
          • dynamic string

            Values are strict, runtime, true, or false.

          • dynamic_date_formats array[string]
          • dynamic_templates array[object]
          • _field_names object
            Hide _field_names attribute Show _field_names attribute object
            • enabled boolean Required
          • index_field object
            Hide index_field attribute Show index_field attribute object
            • enabled boolean Required
          • _meta object
            Hide _meta attribute Show _meta attribute object
            • * object Additional properties
          • numeric_detection boolean
          • properties object
          • _routing object
            Hide _routing attribute Show _routing attribute object
            • required boolean Required
          • _size object
            Hide _size attribute Show _size attribute object
            • enabled boolean Required
          • _source object
            Hide _source attributes Show _source attributes object
            • compress boolean
            • compress_threshold string
            • enabled boolean
            • excludes array[string]
            • includes array[string]
            • mode string

              Values are disabled, stored, or synthetic.

          • runtime object
            Hide runtime attribute Show runtime attribute object
            • * object Additional properties
              Hide * attributes Show * attributes object
              • fields object

                For type composite

              • fetch_fields array[object]

                For type lookup

              • format string

                A custom format for date type runtime fields.

              • input_field string

                Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

              • target_field string

                Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

              • target_index string
              • script object
              • type string Required

                Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

          • enabled boolean
          • subobjects string

            Values are true or false.

          • _data_stream_timestamp object
            Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
            • enabled boolean Required
        • aliases object
          Hide aliases attribute Show aliases attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • filter object

              An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

              External documentation
            • index_routing string

              Value used to route indexing operations to a specific shard. If specified, this overwrites the routing value for indexing operations.

            • is_write_index boolean

              If true, the index is the write index for the alias.

            • routing string

              Value used to route indexing and search operations to a specific shard.

            • search_routing string

              Value used to route search operations to a specific shard. If specified, this overwrites the routing value for search operations.

            • is_hidden boolean Generally available; Added in 7.16.0

              If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

        • lifecycle object

          Data stream lifecycle with rollover can be used to display the configuration including the default rollover conditions, if asked.

          Hide lifecycle attributes Show lifecycle attributes object
          • data_retention string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • downsampling object
            Hide downsampling attribute Show downsampling attribute object
            • rounds array[object] Required

              The list of downsampling rounds to execute as part of this downsampling configuration

          • enabled boolean

            If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

          • rollover object
            Hide rollover attributes Show rollover attributes object
            • min_age string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • max_age string
            • min_docs number
            • max_docs number
            • min_size
            • max_size
            • min_primary_shard_size
            • max_primary_shard_size
            • min_primary_shard_docs number
            • max_primary_shard_docs number
      • version number
      • _meta object
        Hide _meta attribute Show _meta attribute object
        • * object Additional properties
      • deprecated boolean
  • index_template_substitutions object

    A map of index template names to substitute index template definition objects.

    Hide index_template_substitutions attribute Show index_template_substitutions attribute object
    • * object
      Hide * attributes Show * attributes object
      • index_patterns string | array[string] Required
      • composed_of array[string] Required

        An ordered list of component template names. Component templates are merged in the order specified, meaning that the last component template specified has the highest precedence.

      • template object
        Hide template attributes Show template attributes object
        • aliases object

          Aliases to add. If the index template includes a data_stream object, these are data stream aliases. Otherwise, these are index aliases. Data stream aliases ignore the index_routing, routing, and search_routing options.

          Hide aliases attribute Show aliases attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • filter object

              An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

              External documentation
            • index_routing string
            • is_hidden boolean

              If true, the alias is hidden. All indices for the alias must have the same is_hidden value.

            • is_write_index boolean

              If true, the index is the write index for the alias.

            • routing string
            • search_routing string
        • mappings object
          Hide mappings attributes Show mappings attributes object
          • all_field object
            Hide all_field attributes Show all_field attributes object
            • analyzer string Required
            • enabled boolean Required
            • omit_norms boolean Required
            • search_analyzer string Required
            • similarity string Required
            • store boolean Required
            • store_term_vector_offsets boolean Required
            • store_term_vector_payloads boolean Required
            • store_term_vector_positions boolean Required
            • store_term_vectors boolean Required
          • date_detection boolean
          • dynamic string

            Values are strict, runtime, true, or false.

          • dynamic_date_formats array[string]
          • dynamic_templates array[object]
          • _field_names object
            Hide _field_names attribute Show _field_names attribute object
            • enabled boolean Required
          • index_field object
            Hide index_field attribute Show index_field attribute object
            • enabled boolean Required
          • _meta object
            Hide _meta attribute Show _meta attribute object
            • * object Additional properties
          • numeric_detection boolean
          • properties object
          • _routing object
            Hide _routing attribute Show _routing attribute object
            • required boolean Required
          • _size object
            Hide _size attribute Show _size attribute object
            • enabled boolean Required
          • _source object
            Hide _source attributes Show _source attributes object
            • compress boolean
            • compress_threshold string
            • enabled boolean
            • excludes array[string]
            • includes array[string]
            • mode string

              Values are disabled, stored, or synthetic.

          • runtime object
            Hide runtime attribute Show runtime attribute object
            • * object Additional properties
              Hide * attributes Show * attributes object
              • fields object

                For type composite

              • fetch_fields array[object]

                For type lookup

              • format string

                A custom format for date type runtime fields.

              • input_field string

                Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

              • target_field string

                Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

              • target_index string
              • script object
              • type string Required

                Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

          • enabled boolean
          • subobjects string

            Values are true or false.

          • _data_stream_timestamp object
            Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
            • enabled boolean Required
        • settings object
          Index settings
        • lifecycle object

          Data stream lifecycle with rollover can be used to display the configuration including the default rollover conditions, if asked.

          Hide lifecycle attributes Show lifecycle attributes object
          • data_retention string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • downsampling object
            Hide downsampling attribute Show downsampling attribute object
            • rounds array[object] Required

              The list of downsampling rounds to execute as part of this downsampling configuration

          • enabled boolean

            If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

          • rollover object
            Hide rollover attributes Show rollover attributes object
            • min_age string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • max_age string
            • min_docs number
            • max_docs number
            • min_size
            • max_size
            • min_primary_shard_size
            • max_primary_shard_size
            • min_primary_shard_docs number
            • max_primary_shard_docs number
      • version number
      • priority number

        Priority to determine index template precedence when a new data stream or index is created. The index template with the highest priority is chosen. If no priority is specified the template is treated as though it is of priority 0 (lowest priority). This number is not automatically generated by Elasticsearch.

      • _meta object
        Hide _meta attribute Show _meta attribute object
        • * object Additional properties
      • allow_auto_create boolean
      • data_stream object
        Hide data_stream attributes Show data_stream attributes object
        • hidden boolean

          If true, the data stream is hidden.

        • allow_custom_routing boolean

          If true, the data stream supports custom routing.

      • deprecated boolean Generally available; Added in 8.12.0

        Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.

      • ignore_missing_component_templates string | array[string]
  • mapping_addition object
    Hide mapping_addition attributes Show mapping_addition attributes object
    • all_field object
      Hide all_field attributes Show all_field attributes object
      • analyzer string Required
      • enabled boolean Required
      • omit_norms boolean Required
      • search_analyzer string Required
      • similarity string Required
      • store boolean Required
      • store_term_vector_offsets boolean Required
      • store_term_vector_payloads boolean Required
      • store_term_vector_positions boolean Required
      • store_term_vectors boolean Required
    • date_detection boolean
    • dynamic string

      Values are strict, runtime, true, or false.

    • dynamic_date_formats array[string]
    • dynamic_templates array[object]
    • _field_names object
      Hide _field_names attribute Show _field_names attribute object
      • enabled boolean Required
    • index_field object
      Hide index_field attribute Show index_field attribute object
      • enabled boolean Required
    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties
    • numeric_detection boolean
    • properties object
    • _routing object
      Hide _routing attribute Show _routing attribute object
      • required boolean Required
    • _size object
      Hide _size attribute Show _size attribute object
      • enabled boolean Required
    • _source object
      Hide _source attributes Show _source attributes object
      • compress boolean
      • compress_threshold string
      • enabled boolean
      • excludes array[string]
      • includes array[string]
      • mode string

        Values are disabled, stored, or synthetic.

    • runtime object
      Hide runtime attribute Show runtime attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • format string
        • format string

          A custom format for date type runtime fields.

        • input_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_index string
        • script object
          Hide script attributes Show script attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties
          • lang string

            Any of:

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • type string Required

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • enabled boolean
    • subobjects string

      Values are true or false.

    • _data_stream_timestamp object
      Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
      • enabled boolean Required
  • pipeline_substitutions object

    Pipelines to test. If you don’t specify the pipeline request path parameter, this parameter is required. If you specify both this and the request path parameter, the API only uses the request path parameter.

    Hide pipeline_substitutions attribute Show pipeline_substitutions attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • description string

        Description of the ingest pipeline.

      • on_failure array[object]

        Processors to run immediately after a processor failure.

        Hide on_failure attributes Show on_failure attributes object
        • append object
          Hide append attributes Show append attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • value
          • allow_duplicates boolean

            If false, the processor does not append values already present in the field.

        • attachment object
          Hide attachment attributes Show attachment attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • indexed_chars number

            The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

          • indexed_chars_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • remove_binary boolean

            If true, the binary field will be removed from the document

          • resource_name string

            Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

        • bytes object
          Hide bytes attributes Show bytes attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • circle object
          Hide circle attributes Show circle attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • error_distance number Required

            The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • shape_type string Required

            Values are geo_shape or shape.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • community_id object
          Hide community_id attributes Show community_id attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • source_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • source_port string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_port string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • iana_number string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • icmp_type string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • icmp_code string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • transport string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • seed number

            Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • convert object
          Hide convert attributes Show convert attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • type string Required

            Values are integer, long, double, float, boolean, ip, string, or auto.

        • csv object
          Hide csv attributes Show csv attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • empty_value object

            Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • quote string

            Quote used in CSV, has to be single character string.

          • separator string

            Separator used in CSV, has to be single character string.

          • target_fields string | array[string] Required
          • trim boolean

            Trim whitespaces in unquoted fields.

        • date object
          Hide date attributes Show date attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • formats array[string] Required

            An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • locale string

            The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • timezone string

            The timezone to use when parsing the date. Supports template snippets.

          • output_format string

            The format to use when writing the date to target_field. Must be a valid java time pattern.

        • date_index_name object
          Hide date_index_name attributes Show date_index_name attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • date_formats array[string] Required

            An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • date_rounding string Required

            How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • index_name_format string

            The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

          • index_name_prefix string

            A prefix of the index name to be prepended before the printed date. Supports template snippets.

          • locale string

            The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

          • timezone string

            The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

        • dissect object
          Hide dissect attributes Show dissect attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • append_separator string

            The character(s) that separate the appended fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to apply to the field.

        • dot_expander object
          Hide dot_expander attributes Show dot_expander attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • override boolean

            Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

          • path string

            The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

        • drop object
          Hide drop attributes Show drop attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • enrich object
          Hide enrich attributes Show enrich attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • max_matches number

            The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

          • override boolean

            If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • policy_name string Required

            The name of the enrich policy to use.

          • shape_relation string

            Values are intersects, disjoint, within, or contains.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • fail object
          Hide fail attributes Show fail attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • message string Required

            The error message thrown by the processor. Supports template snippets.

        • fingerprint object
          Hide fingerprint attributes Show fingerprint attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • fields string | array[string] Required
          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • salt string

            Salt value for the hash function.

          • method string

            Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

          • ignore_missing boolean

            If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

        • foreach object
          Hide foreach attributes Show foreach attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true, the processor silently exits without changing the document if the field is null or missing.

          • processor object Required
        • ip_location object
          Hide ip_location attributes Show ip_location attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • database_file string

            The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found IP location data will be returned, even if the field contains an array.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the IP location lookup.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • download_database_on_pipeline_creation boolean

            If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • geo_grid object
          Hide geo_grid attributes Show geo_grid attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            The field to interpret as a geo-tile.= The field format is determined by the tile_type.

          • tile_type string Required

            Values are geotile, geohex, or geohash.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • parent_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • children_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • non_children_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • precision_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_format string

            Values are geojson or wkt.

        • geoip object
          Hide geoip attributes Show geoip attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • database_file string

            The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found geoip data will be returned, even if the field contains an array.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the geoip lookup.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • download_database_on_pipeline_creation boolean

            If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • grok object
          Hide grok attributes Show grok attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • ecs_compatibility string

            Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern_definitions object

            A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          • patterns array[string] Required

            An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

          • trace_match boolean

            When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

        • gsub object
          Hide gsub attributes Show gsub attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to be replaced.

          • replacement string Required

            The string to replace the matching patterns with.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • html_strip object
          Hide html_strip attributes Show html_strip attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document,

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • inference object
          Hide inference attributes Show inference attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • model_id string Required
          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_map object

            Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          • inference_config object
          • input_output
          • ignore_missing boolean

            If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

        • join object
          Hide join attributes Show join attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • separator string Required

            The separator character.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • json object
          Hide json attributes Show json attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • add_to_root boolean

            Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

          • add_to_root_conflict_strategy string

            Values are replace or merge.

          • allow_duplicate_keys boolean

            When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • kv object
          Hide kv attributes Show kv attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • exclude_keys array[string]

            List of keys to exclude from document.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_split string Required

            Regex pattern to use for splitting key-value pairs.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • include_keys array[string]

            List of keys to filter and insert into document. Defaults to including all keys.

          • prefix string

            Prefix to be added to extracted keys.

          • strip_brackets boolean

            If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • trim_key string

            String of characters to trim from extracted keys.

          • trim_value string

            String of characters to trim from extracted values.

          • value_split string Required

            Regex pattern to use for splitting the key from the value within a key-value pair.

        • lowercase object
          Hide lowercase attributes Show lowercase attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • network_direction object
          Hide network_direction attributes Show network_direction attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • source_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • internal_networks array[string]

            List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

          • internal_networks_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • pipeline object
          Hide pipeline attributes Show pipeline attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • name string Required
          • ignore_missing_pipeline boolean

            Whether to ignore missing pipelines instead of failing.

        • redact object
          Hide redact attributes Show redact attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • patterns array[string] Required

            A list of grok expressions to match and redact named captures with

          • pattern_definitions object
          • prefix string

            Start a redacted section with this token

          • suffix string

            End a redacted section with this token

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • skip_if_unlicensed boolean

            If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

          • trace_redact boolean Generally available; Added in 8.16.0

            If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

        • registered_domain object
          Hide registered_domain attributes Show registered_domain attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • remove object
          Hide remove attributes Show remove attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string | array[string] Required
          • keep string | array[string]
          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • rename object
          Hide rename attributes Show rename attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • reroute object
          Hide reroute attributes Show reroute attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • destination string

            A static value for the target. Can’t be set when the dataset or namespace option is set.

          • dataset
          • namespace
        • script object
          Hide script attributes Show script attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • id string
          • lang string

            Script language.

          • params object

            Object containing parameters for the script.

          • source string

            Inline script. If no id is specified, this parameter is required.

        • set object
          Hide set attributes Show set attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • copy_from string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_empty_value boolean

            If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

          • media_type string

            The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

          • override boolean

            If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • value object

            The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

        • set_security_user object
          Hide set_security_user attributes Show set_security_user attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what user related properties are added to the field.

        • sort object
          Hide sort attributes Show sort attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • order string

            Values are asc or desc.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • split object
          Hide split attributes Show split attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • preserve_trailing boolean

            Preserves empty trailing fields, if any.

          • separator string Required

            A regex which matches the separator, for example, , or \s+.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • terminate object
          Hide terminate attributes Show terminate attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • trim object
          Hide trim attributes Show trim attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • uppercase object
          Hide uppercase attributes Show uppercase attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • urldecode object
          Hide urldecode attributes Show urldecode attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • uri_parts object
          Hide uri_parts attributes Show uri_parts attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • keep_original boolean

            If true, the processor copies the unparsed URI to <target_field>.original.

          • remove_if_successful boolean

            If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • user_agent object
          Hide user_agent attributes Show user_agent attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • regex_file string

            The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what properties are added to target_field.

            Values are name, os, device, original, or version.

          • extract_device_type boolean Beta; Added in 8.9.0

            Extracts device type from the user agent string on a best-effort basis.

      • processors array[object]

        Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.

        Hide processors attributes Show processors attributes object
        • append object
          Hide append attributes Show append attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • value
          • allow_duplicates boolean

            If false, the processor does not append values already present in the field.

        • attachment object
          Hide attachment attributes Show attachment attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • indexed_chars number

            The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

          • indexed_chars_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • remove_binary boolean

            If true, the binary field will be removed from the document

          • resource_name string

            Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

        • bytes object
          Hide bytes attributes Show bytes attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • circle object
          Hide circle attributes Show circle attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • error_distance number Required

            The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • shape_type string Required

            Values are geo_shape or shape.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • community_id object
          Hide community_id attributes Show community_id attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • source_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • source_port string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_port string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • iana_number string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • icmp_type string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • icmp_code string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • transport string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • seed number

            Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • convert object
          Hide convert attributes Show convert attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • type string Required

            Values are integer, long, double, float, boolean, ip, string, or auto.

        • csv object
          Hide csv attributes Show csv attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • empty_value object

            Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • quote string

            Quote used in CSV, has to be single character string.

          • separator string

            Separator used in CSV, has to be single character string.

          • target_fields string | array[string] Required
          • trim boolean

            Trim whitespaces in unquoted fields.

        • date object
          Hide date attributes Show date attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • formats array[string] Required

            An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • locale string

            The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • timezone string

            The timezone to use when parsing the date. Supports template snippets.

          • output_format string

            The format to use when writing the date to target_field. Must be a valid java time pattern.

        • date_index_name object
          Hide date_index_name attributes Show date_index_name attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • date_formats array[string] Required

            An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • date_rounding string Required

            How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • index_name_format string

            The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

          • index_name_prefix string

            A prefix of the index name to be prepended before the printed date. Supports template snippets.

          • locale string

            The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

          • timezone string

            The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

        • dissect object
          Hide dissect attributes Show dissect attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • append_separator string

            The character(s) that separate the appended fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to apply to the field.

        • dot_expander object
          Hide dot_expander attributes Show dot_expander attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • override boolean

            Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

          • path string

            The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

        • drop object
          Hide drop attributes Show drop attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • enrich object
          Hide enrich attributes Show enrich attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • max_matches number

            The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

          • override boolean

            If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • policy_name string Required

            The name of the enrich policy to use.

          • shape_relation string

            Values are intersects, disjoint, within, or contains.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • fail object
          Hide fail attributes Show fail attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • message string Required

            The error message thrown by the processor. Supports template snippets.

        • fingerprint object
          Hide fingerprint attributes Show fingerprint attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • fields string | array[string] Required
          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • salt string

            Salt value for the hash function.

          • method string

            Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

          • ignore_missing boolean

            If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

        • foreach object
          Hide foreach attributes Show foreach attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true, the processor silently exits without changing the document if the field is null or missing.

          • processor object Required
        • ip_location object
          Hide ip_location attributes Show ip_location attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • database_file string

            The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found IP location data will be returned, even if the field contains an array.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the IP location lookup.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • download_database_on_pipeline_creation boolean

            If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • geo_grid object
          Hide geo_grid attributes Show geo_grid attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            The field to interpret as a geo-tile.= The field format is determined by the tile_type.

          • tile_type string Required

            Values are geotile, geohex, or geohash.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • parent_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • children_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • non_children_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • precision_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_format string

            Values are geojson or wkt.

        • geoip object
          Hide geoip attributes Show geoip attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • database_file string

            The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found geoip data will be returned, even if the field contains an array.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the geoip lookup.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • download_database_on_pipeline_creation boolean

            If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • grok object
          Hide grok attributes Show grok attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • ecs_compatibility string

            Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern_definitions object

            A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          • patterns array[string] Required

            An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

          • trace_match boolean

            When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

        • gsub object
          Hide gsub attributes Show gsub attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to be replaced.

          • replacement string Required

            The string to replace the matching patterns with.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • html_strip object
          Hide html_strip attributes Show html_strip attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document,

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • inference object
          Hide inference attributes Show inference attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • model_id string Required
          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_map object

            Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          • inference_config object
          • input_output
          • ignore_missing boolean

            If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

        • join object
          Hide join attributes Show join attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • separator string Required

            The separator character.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • json object
          Hide json attributes Show json attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • add_to_root boolean

            Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

          • add_to_root_conflict_strategy string

            Values are replace or merge.

          • allow_duplicate_keys boolean

            When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • kv object
          Hide kv attributes Show kv attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • exclude_keys array[string]

            List of keys to exclude from document.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_split string Required

            Regex pattern to use for splitting key-value pairs.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • include_keys array[string]

            List of keys to filter and insert into document. Defaults to including all keys.

          • prefix string

            Prefix to be added to extracted keys.

          • strip_brackets boolean

            If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • trim_key string

            String of characters to trim from extracted keys.

          • trim_value string

            String of characters to trim from extracted values.

          • value_split string Required

            Regex pattern to use for splitting the key from the value within a key-value pair.

        • lowercase object
          Hide lowercase attributes Show lowercase attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • network_direction object
          Hide network_direction attributes Show network_direction attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • source_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • internal_networks array[string]

            List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

          • internal_networks_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • pipeline object
          Hide pipeline attributes Show pipeline attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • name string Required
          • ignore_missing_pipeline boolean

            Whether to ignore missing pipelines instead of failing.

        • redact object
          Hide redact attributes Show redact attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • patterns array[string] Required

            A list of grok expressions to match and redact named captures with

          • pattern_definitions object
          • prefix string

            Start a redacted section with this token

          • suffix string

            End a redacted section with this token

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • skip_if_unlicensed boolean

            If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

          • trace_redact boolean Generally available; Added in 8.16.0

            If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

        • registered_domain object
          Hide registered_domain attributes Show registered_domain attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • remove object
          Hide remove attributes Show remove attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string | array[string] Required
          • keep string | array[string]
          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • rename object
          Hide rename attributes Show rename attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • reroute object
          Hide reroute attributes Show reroute attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • destination string

            A static value for the target. Can’t be set when the dataset or namespace option is set.

          • dataset
          • namespace
        • script object
          Hide script attributes Show script attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • id string
          • lang string

            Script language.

          • params object

            Object containing parameters for the script.

          • source string

            Inline script. If no id is specified, this parameter is required.

        • set object
          Hide set attributes Show set attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • copy_from string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_empty_value boolean

            If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

          • media_type string

            The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

          • override boolean

            If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • value object

            The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

        • set_security_user object
          Hide set_security_user attributes Show set_security_user attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what user related properties are added to the field.

        • sort object
          Hide sort attributes Show sort attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • order string

            Values are asc or desc.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • split object
          Hide split attributes Show split attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • preserve_trailing boolean

            Preserves empty trailing fields, if any.

          • separator string Required

            A regex which matches the separator, for example, , or \s+.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • terminate object
          Hide terminate attributes Show terminate attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • trim object
          Hide trim attributes Show trim attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • uppercase object
          Hide uppercase attributes Show uppercase attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • urldecode object
          Hide urldecode attributes Show urldecode attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • uri_parts object
          Hide uri_parts attributes Show uri_parts attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • keep_original boolean

            If true, the processor copies the unparsed URI to <target_field>.original.

          • remove_if_successful boolean

            If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • user_agent object
          Hide user_agent attributes Show user_agent attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • regex_file string

            The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what properties are added to target_field.

            Values are name, os, device, original, or version.

          • extract_device_type boolean Beta; Added in 8.9.0

            Extracts device type from the user agent string on a best-effort basis.

      • version number
      • deprecated boolean

        Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.

      • _meta object
        Hide _meta attribute Show _meta attribute object
        • * object Additional properties

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required
      Hide docs attribute Show docs attribute object
      • doc object

        The results of ingest simulation on a single document. The _source of the document contains the results after running all pipelines listed in executed_pipelines on the document. The list of executed pipelines is derived from the pipelines that would be executed if this document had been ingested into _index.

        Hide doc attributes Show doc attributes object
        • _id string Required
        • _index string Required
        • _source object Required

          JSON body for the document.

          Hide _source attribute Show _source attribute object
          • * object Additional properties
        • _version number | string Required

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

        • executed_pipelines array[string] Required

          A list of the names of the pipelines executed on this document.

        • ignored_fields array[object]

          A list of the fields that would be ignored at the indexing step. For example, a field whose value is larger than the allowed limit would make it through all of the pipelines, but would not be indexed into Elasticsearch.

          Hide ignored_fields attribute Show ignored_fields attribute object
          • * string Additional properties
        • error object

          Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          Hide error attributes Show error attributes object
          • type string Required

            The type of error

          • reason string | null

            A human-readable explanation of the error, in English.

          • stack_trace string

            The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • caused_by object

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • root_cause array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

          • suppressed array[object]

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

            Cause and details about a request failure. This class defines the properties common to all error types. Additional details are also provided, that depend on the error type.

GET /_ingest/{index}/_simulate
curl \
 --request GET 'http://api.example.com/_ingest/{index}/_simulate' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"docs\": [\n    {\n      \"_id\": 123,\n      \"_index\": \"my-index\",\n      \"_source\": {\n        \"foo\": \"bar\"\n      }\n    },\n    {\n      \"_id\": 456,\n      \"_index\": \"my-index\",\n      \"_source\": {\n        \"foo\": \"rab\"\n      }\n    }\n  ]\n}"'
In this example the index `my-index` has a default pipeline called `my-pipeline` and a final pipeline called `my-final-pipeline`. Since both documents are being ingested into `my-index`, both pipelines are run using the pipeline definitions that are already in the system.
{
  "docs": [
    {
      "_id": 123,
      "_index": "my-index",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_id": 456,
      "_index": "my-index",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}
In this example the index `my-index` has a default pipeline called `my-pipeline` and a final pipeline called `my-final-pipeline`. But a substitute definition of `my-pipeline` is provided in `pipeline_substitutions`. The substitute `my-pipeline` will be used in place of the `my-pipeline` that is in the system, and then the `my-final-pipeline` that is already defined in the system will run.
{
  "docs": [
    {
      "_index": "my-index",
      "_id": 123,
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "my-index",
      "_id": 456,
      "_source": {
        "foo": "rab"
      }
    }
  ],
  "pipeline_substitutions": {
    "my-pipeline": {
      "processors": [
        {
          "uppercase": {
            "field": "foo"
          }
        }
      ]
    }
  }
}
In this example, imagine that the index `my-index` has a strict mapping with only the `foo` keyword field defined. Say that field mapping came from a component template named `my-mappings-template`. You want to test adding a new field, `bar`. So a substitute definition of `my-mappings-template` is provided in `component_template_substitutions`. The substitute `my-mappings-template` will be used in place of the existing mapping for `my-index` and in place of the `my-mappings-template` that is in the system.
{
  "docs": [
    {
      "_index": "my-index",
      "_id": "123",
      "_source": {
        "foo": "foo"
      }
    },
    {
      "_index": "my-index",
      "_id": "456",
      "_source": {
        "bar": "rab"
      }
    }
  ],
  "component_template_substitutions": {
    "my-mappings_template": {
      "template": {
        "mappings": {
          "dynamic": "strict",
          "properties": {
            "foo": {
              "type": "keyword"
            },
            "bar": {
              "type": "keyword"
            }
          }
        }
      }
    }
  }
}
The pipeline, component template, and index template substitutions replace the existing pipeline details for the duration of this request.
{
  "docs": [
    {
      "_id": "id",
      "_index": "my-index",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_id": "id",
      "_index": "my-index",
      "_source": {
        "foo": "rab"
      }
    }
  ],
  "pipeline_substitutions": {
    "my-pipeline": {
      "processors": [
        {
          "set": {
            "field": "field3",
            "value": "value3"
          }
        }
      ]
    }
  },
  "component_template_substitutions": {
    "my-component-template": {
      "template": {
        "mappings": {
          "dynamic": true,
          "properties": {
            "field3": {
              "type": "keyword"
            }
          }
        },
        "settings": {
          "index": {
            "default_pipeline": "my-pipeline"
          }
        }
      }
    }
  },
  "index_template_substitutions": {
    "my-index-template": {
      "index_patterns": [
        "my-index-*"
      ],
      "composed_of": [
        "component_template_1",
        "component_template_2"
      ]
    }
  },
  "mapping_addition": {
    "dynamic": "strict",
    "properties": {
      "foo": {
        "type": "keyword"
      }
    }
  }
}
A successful response when the simulation uses pipeline definitions that are already in the system.
{
  "docs": [
    {
      "doc": null,
      "_id": 123,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field1": "value1",
        "field2": "value2",
        "foo": "bar"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    },
    {
      "doc": null,
      "_id": 456,
      "_index": "my-index",
      "_version": "-3,",
      "_source": {
        "field1": "value1",
        "field2": "value2",
        "foo": "rab"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    }
  ]
}
A successful response when the simulation uses pipeline substitutions.
{
  "docs": [
    {
      "doc": null,
      "_id": 123,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field2": "value2",
        "foo": "BAR"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    },
    {
      "doc": null,
      "_id": 456,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field2": "value2",
        "foo": "RAB"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    }
  ]
}
A successful response when the simulation uses pipeline substitutions.
{
  "docs": [
    {
      "doc": {
        "_id": "123",
        "_index": "my-index",
        "_version": -3,
        "_source": {
          "foo": "foo"
        },
        "executed_pipelines": []
      }
    },
    {
      "doc": {
        "_id": "456",
        "_index": "my-index",
        "_version": -3,
        "_source": {
          "bar": "rab"
        },
      "executed_pipelines": []
      }
    }
  ]
}
























Get the trial status Generally available; Added in 6.1.0

GET /_license/trial_status

Required authorization

  • Cluster privileges: monitor

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • eligible_to_start_trial boolean Required
GET /_license/trial_status
curl \
 --request GET 'http://api.example.com/_license/trial_status' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_license/trial_status`.
{
  "eligible_to_start_trial": true
}









Get Logstash pipelines Generally available; Added in 7.12.0

GET /_logstash/pipeline/{id}

Get pipelines that are used for Logstash Central Management.

Required authorization

  • Cluster privileges: manage_logstash_pipelines
External documentation

Path parameters

  • id string | array[string] Required

    A comma-separated list of pipeline identifiers.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • description string Required

        A description of the pipeline. This description is not used by Elasticsearch or Logstash.

      • last_modified string | number Required

        A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

        One of:
      • pipeline string Required

        The configuration for the pipeline.

        External documentation
      • pipeline_metadata object Required
        Hide pipeline_metadata attributes Show pipeline_metadata attributes object
        • type string Required
        • version string Required
      • pipeline_settings object Required
        Hide pipeline_settings attributes Show pipeline_settings attributes object
        • pipeline.workers number Required

          The number of workers that will, in parallel, execute the filter and output stages of the pipeline.

        • pipeline.batch.size number Required

          The maximum number of events an individual worker thread will collect from inputs before attempting to execute its filters and outputs.

        • pipeline.batch.delay number Required

          When creating pipeline event batches, how long in milliseconds to wait for each event before dispatching an undersized batch to pipeline workers.

        • queue.type string Required

          The internal queuing model to use for event buffering.

        • queue.max_bytes.number number Required

          The total capacity of the queue (queue.type: persisted) in number of bytes.

        • queue.max_bytes.units string Required

          The total capacity of the queue (queue.type: persisted) in terms of units of bytes.

        • queue.checkpoint.writes number Required

          The maximum number of written events before forcing a checkpoint when persistent queues are enabled (queue.type: persisted).

      • username string Required

        The user who last updated the pipeline.

GET /_logstash/pipeline/{id}
curl \
 --request GET 'http://api.example.com/_logstash/pipeline/{id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _logstash/pipeline/my_pipeline`.
{
  "my_pipeline": {
    "description": "Sample pipeline for illustration purposes",
    "last_modified": "2021-01-02T02:50:51.250Z",
    "pipeline_metadata": {
      "type": "logstash_pipeline",
      "version": "1"
    },
    "username": "elastic",
    "pipeline": "input {}\\n filter { grok {} }\\n output {}",
    "pipeline_settings": {
      "pipeline.workers": 1,
      "pipeline.batch.size": 125,
      "pipeline.batch.delay": 50,
      "queue.type": "memory",
      "queue.max_bytes": "1gb",
      "queue.checkpoint.writes": 1024
    }
  }
}












Machine learning









Get machine learning information Generally available; Added in 6.3.0

GET /_ml/info

Get defaults and limits used by machine learning. This endpoint is designed to be used by a user interface that needs to fully understand machine learning configurations where some options are not specified, meaning that the defaults should be used. This endpoint may be used to find out what those defaults are. It also provides information about the maximum size of machine learning jobs that could run in the current cluster configuration.

Required authorization

  • Cluster privileges: monitor_ml

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • defaults object Required
      Hide defaults attributes Show defaults attributes object
      • anomaly_detectors object Required
        Hide anomaly_detectors attributes Show anomaly_detectors attributes object
        • categorization_analyzer string | object Required

          One of:
        • categorization_examples_limit number Required
        • model_memory_limit string Required
        • model_snapshot_retention_days number Required
        • daily_model_snapshot_retention_after_days number Required
      • datafeeds object Required
        Hide datafeeds attribute Show datafeeds attribute object
        • scroll_size number Required
    • limits object Required
      Hide limits attributes Show limits attributes object
    • upgrade_mode boolean Required
    • native_code object Required
      Hide native_code attributes Show native_code attributes object
      • build_hash string Required
      • version string Required
GET /_ml/info
curl \
 --request GET 'http://api.example.com/_ml/info' \
 --header "Authorization: $API_KEY"




Machine learning anomaly detection

Close anomaly detection jobs Generally available; Added in 5.4.0

POST /_ml/anomaly_detectors/{job_id}/_close

A job can be opened and closed multiple times throughout its lifecycle. A closed job cannot receive data or perform analysis operations, but you can still explore and navigate results. When you close a job, it runs housekeeping tasks such as pruning the model history, flushing buffers, calculating final results and persisting the model snapshots. Depending upon the size of the job, it could take several minutes to close and the equivalent time to re-open. After it is closed, the job has a minimal overhead on the cluster except for maintaining its meta data. Therefore it is a best practice to close jobs that are no longer required to process data. If you close an anomaly detection job whose datafeed is running, the request first tries to stop the datafeed. This behavior is equivalent to calling stop datafeed API with the same timeout and force parameters as the close job request. When a datafeed that has a specified end date stops, it automatically closes its associated job.

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job. It can be a job identifier, a group name, or a wildcard expression. You can close multiple anomaly detection jobs in a single API request by using a group name, a comma-separated list of jobs, or a wildcard expression. You can close all jobs by using _all or by specifying * as the job identifier.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request: contains wildcard expressions and there are no jobs that match; contains the _all string or no identifiers and there are no matches; or contains wildcard expressions and there are only partial matches. By default, it returns an empty jobs array when there are no matches and the subset of results when there are partial matches. If false, the request returns a 404 status code when there are no matches or only partial matches.

  • force boolean

    Use to close a failed job, or to forcefully close a job which has not responded to its initial close request; the request returns without performing the associated actions such as flushing buffers and persisting the model snapshots. If you want the job to be in a consistent state after the close job API returns, do not set to true. This parameter should be used only in situations where the job has already failed or where you are not interested in results the job might have recently produced or might produce in the future.

  • timeout string

    Controls the time to wait until a job has closed.

    Values are -1 or 0.

application/json

Body

  • allow_no_match boolean

    Refer to the description for the allow_no_match query parameter.

  • force boolean

    Refer to the descriptiion for the force query parameter.

  • timeout string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • closed boolean Required
POST /_ml/anomaly_detectors/{job_id}/_close
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_close' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"allow_no_match":true,"force":true,"timeout":"string"}'
Response examples (200)
A successful response when closing anomaly detection jobs.
{
  "closed": true
}

Get calendar configuration info Generally available; Added in 6.2.0

GET /_ml/calendars/{calendar_id}

Required authorization

  • Cluster privileges: monitor_ml

Path parameters

  • calendar_id string Required

    A string that uniquely identifies a calendar. You can get information for multiple calendars by using a comma-separated list of ids or a wildcard expression. You can get information for all calendars by using _all or * or by omitting the calendar identifier.

Query parameters

  • from number

    Skips the specified number of calendars. This parameter is supported only when you omit the calendar identifier.

  • size number

    Specifies the maximum number of calendars to obtain. This parameter is supported only when you omit the calendar identifier.

application/json

Body

  • page object
    Hide page attributes Show page attributes object
    • from number

      Skips the specified number of items.

    • size number

      Specifies the maximum number of items to obtain.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • calendars array[object] Required
      Hide calendars attributes Show calendars attributes object
      • calendar_id string Required
      • description string

        A description of the calendar.

      • job_ids array[string] Required

        An array of anomaly detection job identifiers.

    • count number Required
GET /_ml/calendars/{calendar_id}
curl \
 --request GET 'http://api.example.com/_ml/calendars/{calendar_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"page":{"from":42.0,"size":42.0}}'

Create a calendar Generally available; Added in 6.2.0

PUT /_ml/calendars/{calendar_id}

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • calendar_id string Required

    A string that uniquely identifies a calendar.

application/json

Body

  • job_ids array[string]

    An array of anomaly detection job identifiers.

  • description string

    A description of the calendar.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
PUT /_ml/calendars/{calendar_id}
curl \
 --request PUT 'http://api.example.com/_ml/calendars/{calendar_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"job_ids":["string"],"description":"string"}'

Get calendar configuration info Generally available; Added in 6.2.0

POST /_ml/calendars/{calendar_id}

Required authorization

  • Cluster privileges: monitor_ml

Path parameters

  • calendar_id string Required

    A string that uniquely identifies a calendar. You can get information for multiple calendars by using a comma-separated list of ids or a wildcard expression. You can get information for all calendars by using _all or * or by omitting the calendar identifier.

Query parameters

  • from number

    Skips the specified number of calendars. This parameter is supported only when you omit the calendar identifier.

  • size number

    Specifies the maximum number of calendars to obtain. This parameter is supported only when you omit the calendar identifier.

application/json

Body

  • page object
    Hide page attributes Show page attributes object
    • from number

      Skips the specified number of items.

    • size number

      Specifies the maximum number of items to obtain.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • calendars array[object] Required
      Hide calendars attributes Show calendars attributes object
      • calendar_id string Required
      • description string

        A description of the calendar.

      • job_ids array[string] Required

        An array of anomaly detection job identifiers.

    • count number Required
POST /_ml/calendars/{calendar_id}
curl \
 --request POST 'http://api.example.com/_ml/calendars/{calendar_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"page":{"from":42.0,"size":42.0}}'

Delete a calendar Generally available; Added in 6.2.0

DELETE /_ml/calendars/{calendar_id}

Remove all scheduled events from a calendar, then delete it.

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • calendar_id string Required

    A string that uniquely identifies a calendar.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

DELETE /_ml/calendars/{calendar_id}
curl \
 --request DELETE 'http://api.example.com/_ml/calendars/{calendar_id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response when deleting a calendar.
{
  "acknowledged": true
}












































Predict future behavior of a time series Generally available; Added in 6.1.0

POST /_ml/anomaly_detectors/{job_id}/_forecast

Forecasts are not supported for jobs that perform population analysis; an error occurs if you try to create a forecast for a job that has an over_field_name in its configuration. Forcasts predict future behavior based on historical data.

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job. The job must be open when you create a forecast; otherwise, an error occurs.

Query parameters

  • duration string

    A period of time that indicates how far into the future to forecast. For example, 30d corresponds to 30 days. The forecast starts at the last record that was processed.

    Values are -1 or 0.

  • expires_in string

    The period of time that forecast results are retained. After a forecast expires, the results are deleted. If set to a value of 0, the forecast is never automatically deleted.

    Values are -1 or 0.

  • max_model_memory string

    The maximum memory the forecast can use. If the forecast needs to use more than the provided amount, it will spool to disk. Default is 20mb, maximum is 500mb and minimum is 1mb. If set to 40% or more of the job’s configured memory limit, it is automatically reduced to below that amount.

application/json

Body

  • duration string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • expires_in string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • max_model_memory string

    Refer to the description for the max_model_memory query parameter.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required
    • forecast_id string Required
POST /_ml/anomaly_detectors/{job_id}/_forecast
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_forecast' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"duration\": \"10d\"\n}"'
Request example
An example body for a `POST _ml/anomaly_detectors/low_request_rate/_forecast` request.
{
  "duration": "10d"
}




















Get model snapshots info Generally available; Added in 5.4.0

GET /_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}

Required authorization

  • Cluster privileges: monitor_ml

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

  • snapshot_id string Required

    A numerical character string that uniquely identifies the model snapshot. You can get information for multiple snapshots by using a comma-separated list or a wildcard expression. You can get all snapshots by using _all, by specifying * as the snapshot ID, or by omitting the snapshot ID.

Query parameters

  • desc boolean

    If true, the results are sorted in descending order.

  • end string | number

    Returns snapshots with timestamps earlier than this time.

  • from number

    Skips the specified number of snapshots.

  • size number

    Specifies the maximum number of snapshots to obtain.

  • sort string

    Specifies the sort field for the requested snapshots. By default, the snapshots are sorted by their timestamp.

  • start string | number

    Returns snapshots with timestamps after this time.

application/json

Body

  • desc boolean

    Refer to the description for the desc query parameter.

  • end string | number

    A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

    One of:
  • page object
    Hide page attributes Show page attributes object
    • from number

      Skips the specified number of items.

    • size number

      Specifies the maximum number of items to obtain.

  • sort string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • start string | number

    A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

    One of:

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • model_snapshots array[object] Required
      Hide model_snapshots attributes Show model_snapshots attributes object
      • description string

        An optional description of the job.

      • job_id string Required
      • latest_record_time_stamp number

        The timestamp of the latest processed record.

      • latest_result_time_stamp number

        The timestamp of the latest bucket result.

      • min_version string Required
      • model_size_stats object
        Hide model_size_stats attributes Show model_size_stats attributes object
        • bucket_allocation_failures_count number Required
        • job_id string Required
        • log_time string | number Required

          A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

          One of:
        • memory_status string Required

          Values are ok, soft_limit, or hard_limit.

        • model_bytes number | string Required

        • model_bytes_exceeded number | string

        • model_bytes_memory_limit number | string

        • output_memory_allocator_bytes number | string

        • peak_model_bytes number | string

        • assignment_memory_basis string
        • result_type string Required
        • total_by_field_count number Required
        • total_over_field_count number Required
        • total_partition_field_count number Required
        • categorization_status string Required

          Values are ok or warn.

        • categorized_doc_count number Required
        • dead_category_count number Required
        • failed_category_count number Required
        • frequent_category_count number Required
        • rare_category_count number Required
        • total_category_count number Required
        • timestamp number
      • retain boolean Required

        If true, this snapshot will not be deleted during automatic cleanup of snapshots older than model_snapshot_retention_days. However, this snapshot will be deleted when the job is deleted. The default value is false.

      • snapshot_doc_count number Required

        For internal use only.

      • snapshot_id string Required
      • timestamp number Required

        The creation timestamp for the snapshot.

GET /_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}
curl \
 --request GET 'http://api.example.com/_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"start\": \"1575402236000\"\n}"'
Request example
An example body for a `GET _ml/anomaly_detectors/high_sum_total_sales/model_snapshots` request.
{
  "start": "1575402236000"
}












Force buffered data to be processed Generally available; Added in 5.4.0

POST /_ml/anomaly_detectors/{job_id}/_flush

The flush jobs API is only applicable when sending data for analysis using the post data API. Depending on the content of the buffer, then it might additionally calculate new results. Both flush and close operations are similar, however the flush is more efficient if you are expecting to send more data for analysis. When flushing, the job remains open and is available to continue analyzing data. A close operation additionally prunes and persists the model state to disk and the job must be opened again before analyzing further data.

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

Query parameters

  • advance_time string | number

    Specifies to advance to a particular time value. Results are generated and the model is updated for data from the specified time interval.

  • calc_interim boolean

    If true, calculates the interim results for the most recent bucket or all buckets within the latency period.

  • end string | number

    When used in conjunction with calc_interim and start, specifies the range of buckets on which to calculate interim results.

  • skip_time string | number

    Specifies to skip to a particular time value. Results are not generated and the model is not updated for data from the specified time interval.

  • start string | number

    When used in conjunction with calc_interim, specifies the range of buckets on which to calculate interim results.

application/json

Body

  • advance_time string | number

    A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

    One of:
  • calc_interim boolean

    Refer to the description for the calc_interim query parameter.

  • end string | number

    A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

    One of:
  • skip_time string | number

    A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

    One of:
  • start string | number

    A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

    One of:

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • flushed boolean Required
    • last_finalized_bucket_end number

      Provides the timestamp (in milliseconds since the epoch) of the end of the last bucket that was processed.

POST /_ml/anomaly_detectors/{job_id}/_flush
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_flush' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"calc_interim\": true\n}"'
Request example
An example body for a `POST _ml/anomaly_detectors/low_request_rate/_flush` request.
{
  "calc_interim": true
}








































































Get anomaly detection job stats Generally available; Added in 5.5.0

GET /_ml/anomaly_detectors/_stats

Required authorization

  • Cluster privileges: monitor_ml

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    1. Contains wildcard expressions and there are no jobs that match.
    2. Contains the _all string or no identifiers and there are no matches.
    3. Contains wildcard expressions and there are only partial matches.

    If true, the API returns an empty jobs array when there are no matches and the subset of results when there are partial matches. If false, the API returns a 404 status code when there are no matches or only partial matches.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • jobs array[object] Required
      Hide jobs attributes Show jobs attributes object
      • assignment_explanation string

        For open anomaly detection jobs only, contains messages relating to the selection of a node to run the job.

      • data_counts object Required
        Hide data_counts attributes Show data_counts attributes object
        • bucket_count number Required
        • earliest_record_timestamp number
        • empty_bucket_count number Required
        • input_bytes number Required
        • input_field_count number Required
        • input_record_count number Required
        • invalid_date_count number Required
        • job_id string Required
        • last_data_time number
        • latest_empty_bucket_timestamp number
        • latest_record_timestamp number
        • latest_sparse_bucket_timestamp number
        • latest_bucket_timestamp number
        • log_time number
        • missing_field_count number Required
        • out_of_order_timestamp_count number Required
        • processed_field_count number Required
        • processed_record_count number Required
        • sparse_bucket_count number Required
      • forecasts_stats object Required
        Hide forecasts_stats attributes Show forecasts_stats attributes object
        • memory_bytes object
          Hide memory_bytes attributes Show memory_bytes attributes object
          • avg number Required
          • max number Required
          • min number Required
          • total number Required
        • processing_time_ms object
          Hide processing_time_ms attributes Show processing_time_ms attributes object
          • avg number Required
          • max number Required
          • min number Required
          • total number Required
        • records object
          Hide records attributes Show records attributes object
          • avg number Required
          • max number Required
          • min number Required
          • total number Required
        • status object
          Hide status attribute Show status attribute object
          • * number Additional properties
        • total number Required
        • forecasted_jobs number Required
      • job_id string Required

        Identifier for the anomaly detection job.

      • model_size_stats object Required
        Hide model_size_stats attributes Show model_size_stats attributes object
        • bucket_allocation_failures_count number Required
        • job_id string Required
        • log_time string | number Required

          A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

          One of:
        • memory_status string Required

          Values are ok, soft_limit, or hard_limit.

        • model_bytes number | string Required

        • model_bytes_exceeded number | string

        • model_bytes_memory_limit number | string

        • output_memory_allocator_bytes number | string

        • peak_model_bytes number | string

        • assignment_memory_basis string
        • result_type string Required
        • total_by_field_count number Required
        • total_over_field_count number Required
        • total_partition_field_count number Required
        • categorization_status string Required

          Values are ok or warn.

        • categorized_doc_count number Required
        • dead_category_count number Required
        • failed_category_count number Required
        • frequent_category_count number Required
        • rare_category_count number Required
        • total_category_count number Required
        • timestamp number
      • node object

        Alternative representation of DiscoveryNode used in ml.get_job_stats and ml.get_datafeed_stats

        Hide node attributes Show node attributes object
        • name string Required
        • ephemeral_id string Required
        • id string Required
        • transport_address string Required
        • attributes object Required
          Hide attributes attribute Show attributes attribute object
          • * string Additional properties
      • open_time string | number

        A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

        One of:
      • state string Required

        Values are closing, closed, opened, failed, or opening.

      • timing_stats object Required
        Hide timing_stats attributes Show timing_stats attributes object
        • average_bucket_processing_time_ms number

          Time unit for fractional milliseconds

        • bucket_count number Required
        • exponential_average_bucket_processing_time_ms number

          Time unit for fractional milliseconds

        • exponential_average_bucket_processing_time_per_hour_ms number

          Time unit for fractional milliseconds

        • job_id string Required
        • total_bucket_processing_time_ms number

          Time unit for fractional milliseconds

        • maximum_bucket_processing_time_ms number

          Time unit for fractional milliseconds

        • minimum_bucket_processing_time_ms number

          Time unit for fractional milliseconds

      • deleting boolean

        Indicates that the process of deleting the job is in progress but not yet completed. It is only reported when true.

GET /_ml/anomaly_detectors/_stats
curl \
 --request GET 'http://api.example.com/_ml/anomaly_detectors/_stats' \
 --header "Authorization: $API_KEY"








Get anomaly detection job model snapshot upgrade usage info Generally available; Added in 7.16.0

GET /_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_upgrade/_stats

Required authorization

  • Cluster privileges: monitor_ml

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

  • snapshot_id string Required

    A numerical character string that uniquely identifies the model snapshot. You can get information for multiple snapshots by using a comma-separated list or a wildcard expression. You can get all snapshots by using _all, by specifying * as the snapshot ID, or by omitting the snapshot ID.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    • Contains wildcard expressions and there are no jobs that match.
    • Contains the _all string or no identifiers and there are no matches.
    • Contains wildcard expressions and there are only partial matches.

    The default value is true, which returns an empty jobs array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • model_snapshot_upgrades array[object] Required
      Hide model_snapshot_upgrades attributes Show model_snapshot_upgrades attributes object
      • job_id string Required
      • snapshot_id string Required
      • state string Required

        Values are loading_old_state, saving_new_state, stopped, or failed.

      • node object Required
        Hide node attribute Show node attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • name string
          • ephemeral_id string Required
          • transport_address string Required
          • external_id string Required
          • attributes object Required
            Hide attributes attribute Show attributes attribute object
            • * string Additional properties
          • roles array[string] Required
          • version string Required
          • min_index_version number Required
          • max_index_version number Required
      • assignment_explanation string Required
GET /_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_upgrade/_stats
curl \
 --request GET 'http://api.example.com/_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_upgrade/_stats' \
 --header "Authorization: $API_KEY"












Get overall bucket results Generally available; Added in 6.1.0

POST /_ml/anomaly_detectors/{job_id}/results/overall_buckets

Retrievs overall bucket results that summarize the bucket results of multiple anomaly detection jobs.

The overall_score is calculated by combining the scores of all the buckets within the overall bucket span. First, the maximum anomaly_score per anomaly detection job in the overall bucket is calculated. Then the top_n of those scores are averaged to result in the overall_score. This means that you can fine-tune the overall_score so that it is more or less sensitive to the number of jobs that detect an anomaly at the same time. For example, if you set top_n to 1, the overall_score is the maximum bucket score in the overall bucket. Alternatively, if you set top_n to the number of jobs, the overall_score is high only when all jobs detect anomalies in that overall bucket. If you set the bucket_span parameter (to a value greater than its default), the overall_score is the maximum overall_score of the overall buckets that have a span equal to the jobs' largest bucket span.

Required authorization

  • Cluster privileges: monitor_ml

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job. It can be a job identifier, a group name, a comma-separated list of jobs or groups, or a wildcard expression.

    You can summarize the bucket results for all anomaly detection jobs by using _all or by specifying * as the <job_id>.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    1. Contains wildcard expressions and there are no jobs that match.
    2. Contains the _all string or no identifiers and there are no matches.
    3. Contains wildcard expressions and there are only partial matches.

    If true, the request returns an empty jobs array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.

  • bucket_span string

    The span of the overall buckets. Must be greater or equal to the largest bucket span of the specified anomaly detection jobs, which is the default value.

    By default, an overall bucket has a span equal to the largest bucket span of the specified anomaly detection jobs. To override that behavior, use the optional bucket_span parameter.

    Values are -1 or 0.

  • end string | number

    Returns overall buckets with timestamps earlier than this time.

  • exclude_interim boolean

    If true, the output excludes interim results.

  • overall_score number | string

    Returns overall buckets with overall scores greater than or equal to this value.

  • start string | number

    Returns overall buckets with timestamps after this time.

  • top_n number

    The number of top anomaly detection job bucket scores to be used in the overall_score calculation.

application/json

Body

  • allow_no_match boolean

    Refer to the description for the allow_no_match query parameter.

  • bucket_span string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • end string | number

    A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

    One of:
  • exclude_interim boolean

    Refer to the description for the exclude_interim query parameter.

  • overall_score number | string

    Refer to the description for the overall_score query parameter.

  • start string | number

    A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

    One of:
  • top_n number

    Refer to the description for the top_n query parameter.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • overall_buckets array[object] Required

      Array of overall bucket objects

      Hide overall_buckets attributes Show overall_buckets attributes object
      • bucket_span number

        Time unit for seconds

      • is_interim boolean Required

        If true, this is an interim result. In other words, the results are calculated based on partial input data.

      • jobs array[object] Required

        An array of objects that contain the max_anomaly_score per job_id.

        Hide jobs attributes Show jobs attributes object
        • job_id string Required
        • max_anomaly_score number Required
      • overall_score number Required

        The top_n average of the maximum bucket anomaly_score per job.

      • result_type string Required

        Internal. This is always set to overall_bucket.

      • timestamp number

        Time unit for milliseconds

      • timestamp_string string | number

        A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

        One of:
POST /_ml/anomaly_detectors/{job_id}/results/overall_buckets
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/results/overall_buckets' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"overall_score\": 80,\n  \"start\": \"1403532000000\"\n}"'
Request example
An example body for a `GET _ml/anomaly_detectors/job-*/results/overall_buckets` request.
{
  "overall_score": 80,
  "start": "1403532000000"
}












Send data to an anomaly detection job for analysis Deprecated Generally available; Added in 5.4.0

POST /_ml/anomaly_detectors/{job_id}/_data

IMPORTANT: For each job, data can be accepted from only a single connection at a time. It is not currently possible to post data to multiple jobs using wildcards or a comma-separated list.

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job. The job must have a state of open to receive and process the data.

Query parameters

  • reset_end string | number

    Specifies the end of the bucket resetting range.

  • reset_start string | number

    Specifies the start of the bucket resetting range.

application/json

Body Required

object object

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • job_id string Required
    • processed_record_count number Required
    • processed_field_count number Required
    • input_bytes number Required
    • input_field_count number Required
    • invalid_date_count number Required
    • missing_field_count number Required
    • out_of_order_timestamp_count number Required
    • empty_bucket_count number Required
    • sparse_bucket_count number Required
    • bucket_count number Required
    • earliest_record_timestamp number

      Time unit for milliseconds

    • latest_record_timestamp number

      Time unit for milliseconds

    • last_data_time number

      Time unit for milliseconds

    • latest_empty_bucket_timestamp number

      Time unit for milliseconds

    • latest_sparse_bucket_timestamp number

      Time unit for milliseconds

    • input_record_count number Required
    • log_time number

      Time unit for milliseconds

POST /_ml/anomaly_detectors/{job_id}/_data
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/_data' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '[{}]'












Preview a datafeed Generally available; Added in 5.4.0

POST /_ml/datafeeds/_preview

This API returns the first "page" of search results from a datafeed. You can preview an existing datafeed or provide configuration details for a datafeed and anomaly detection job in the API. The preview shows the structure of the data that will be passed to the anomaly detection engine. IMPORTANT: When Elasticsearch security features are enabled, the preview uses the credentials of the user that called the API. However, when the datafeed starts it uses the roles of the last user that created or updated the datafeed. To get a preview that accurately reflects the behavior of the datafeed, use the appropriate credentials. You can also use secondary authorization headers to supply the credentials.

Required authorization

  • Index privileges: read
  • Cluster privileges: manage_ml

Query parameters

  • start string | number

    The start time from where the datafeed preview should begin

  • end string | number

    The end time when the datafeed preview should stop

application/json

Body

  • datafeed_config object
    Hide datafeed_config attributes Show datafeed_config attributes object
    • aggregations object

      If set, the datafeed performs aggregation searches. Support for aggregations is limited and should be used only with low cardinality data.

    • chunking_config object
      Hide chunking_config attributes Show chunking_config attributes object
      • mode string Required

        Values are auto, manual, or off.

      • time_span string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • datafeed_id string
    • delayed_data_check_config object
      Hide delayed_data_check_config attributes Show delayed_data_check_config attributes object
      • check_window string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • enabled boolean Required

        Specifies whether the datafeed periodically checks for delayed data.

    • frequency string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • indices string | array[string]
    • indices_options object

      Controls how to deal with unavailable concrete indices (closed or missing), how wildcard expressions are expanded to actual indices (all, closed or open indices) and how to deal with wildcard expressions that resolve to no indices.

      Hide indices_options attributes Show indices_options attributes object
      • allow_no_indices boolean

        If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

      • expand_wildcards string | array[string]
      • ignore_unavailable boolean

        If true, missing or closed indices are not included in the response.

      • ignore_throttled boolean

        If true, concrete, expanded or aliased indices are ignored when frozen.

    • job_id string
    • max_empty_searches number

      If a real-time datafeed has never seen any data (including during any initial training period) then it will automatically stop itself and close its associated job after this many real-time searches that return no documents. In other words, it will stop after frequency times max_empty_searches of real-time operation. If not set then a datafeed with no end time that sees no data will remain started until it is explicitly stopped.

    • query object

      An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

      External documentation
    • query_delay string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • runtime_mappings object
      Hide runtime_mappings attribute Show runtime_mappings attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • format string
        • format string

          A custom format for date type runtime fields.

        • input_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_index string
        • script object
          Hide script attributes Show script attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties
          • lang string

            Any of:

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • type string Required

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • script_fields object

      Specifies scripts that evaluate custom expressions and returns script fields to the datafeed. The detector configuration objects in a job can contain functions that use these script fields.

      Hide script_fields attribute Show script_fields attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • script object Required
          Hide script attributes Show script attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties
          • lang string

            Any of:

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • ignore_failure boolean
    • scroll_size number

      The size parameter that is used in Elasticsearch searches when the datafeed does not use aggregations. The maximum value is the value of index.max_result_window, which is 10,000 by default.

  • job_config object
    Hide job_config attributes Show job_config attributes object
    • allow_lazy_open boolean

      Advanced configuration option. Specifies whether this job can open when there is insufficient machine learning node capacity for it to be immediately assigned to a node.

    • analysis_config object Required
      Hide analysis_config attributes Show analysis_config attributes object
      • bucket_span string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • categorization_analyzer string | object

        One of:
      • categorization_field_name string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • categorization_filters array[string]

        If categorization_field_name is specified, you can also define optional filters. This property expects an array of regular expressions. The expressions are used to filter out matching sequences from the categorization field values. You can use this functionality to fine tune the categorization by excluding sequences from consideration when categories are defined. For example, you can exclude SQL statements that appear in your log files. This property cannot be used at the same time as categorization_analyzer. If you only want to define simple regular expression filters that are applied prior to tokenization, setting this property is the easiest method. If you also want to customize the tokenizer or post-tokenization filtering, use the categorization_analyzer property instead and include the filters as pattern_replace character filters. The effect is exactly the same.

      • detectors array[object] Required

        Detector configuration objects specify which data fields a job analyzes. They also specify which analytical functions are used. You can specify multiple detectors for a job. If the detectors array does not contain at least one detector, no analysis can occur and an error is returned.

        Hide detectors attributes Show detectors attributes object
        • by_field_name string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • custom_rules array[object]

          Custom rules enable you to customize the way detectors operate. For example, a rule may dictate conditions under which results should be skipped. Kibana refers to custom rules as job rules.

          Hide custom_rules attributes Show custom_rules attributes object
          • actions array[string]

            The set of actions to be triggered when the rule applies. If more than one action is specified the effects of all actions are combined.

            Values are skip_result or skip_model_update.

          • conditions array[object]

            An array of numeric conditions when the rule applies. A rule must either have a non-empty scope or at least one condition. Multiple conditions are combined together with a logical AND.

          • scope object

            A scope of series where the rule applies. A rule must either have a non-empty scope or at least one condition. By default, the scope includes all series. Scoping is allowed for any of the fields that are also specified in by_field_name, over_field_name, or partition_field_name.

        • detector_description string

          A description of the detector.

        • detector_index number

          A unique identifier for the detector. This identifier is based on the order of the detectors in the analysis_config, starting at zero. If you specify a value for this property, it is ignored.

        • exclude_frequent string

          Values are all, none, by, or over.

        • field_name string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • function string

          The analysis function that is used. For example, count, rare, mean, min, max, or sum.

        • over_field_name string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • partition_field_name string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • use_null boolean

          Defines whether a new series is used as the null series when there is no value for the by or partition fields.

      • influencers array[string]

        A comma separated list of influencer field names. Typically these can be the by, over, or partition fields that are used in the detector configuration. You might also want to use a field name that is not specifically named in a detector, but is available as part of the input data. When you use multiple detectors, the use of influencers is recommended as it aggregates results for each influencer entity.

      • latency string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • model_prune_window string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • multivariate_by_fields boolean

        This functionality is reserved for internal use. It is not supported for use in customer environments and is not subject to the support SLA of official GA features. If set to true, the analysis will automatically find correlations between metrics for a given by field value and report anomalies when those correlations cease to hold. For example, suppose CPU and memory usage on host A is usually highly correlated with the same metrics on host B. Perhaps this correlation occurs because they are running a load-balanced application. If you enable this property, anomalies will be reported when, for example, CPU usage on host A is high and the value of CPU usage on host B is low. That is to say, you’ll see an anomaly when the CPU of host A is unusual given the CPU of host B. To use the multivariate_by_fields property, you must also specify by_field_name in your detector.

      • per_partition_categorization object
        Hide per_partition_categorization attributes Show per_partition_categorization attributes object
        • enabled boolean

          To enable this setting, you must also set the partition_field_name property to the same value in every detector that uses the keyword mlcategory. Otherwise, job creation fails.

        • stop_on_warn boolean

          This setting can be set to true only if per-partition categorization is enabled. If true, both categorization and subsequent anomaly detection stops for partitions where the categorization status changes to warn. This setting makes it viable to have a job where it is expected that categorization works well for some partitions but not others; you do not pay the cost of bad categorization forever in the partitions where it works badly.

      • summary_count_field_name string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • analysis_limits object
      Hide analysis_limits attributes Show analysis_limits attributes object
      • categorization_examples_limit number

        The maximum number of examples stored per category in memory and in the results data store. If you increase this value, more examples are available, however it requires that you have more storage available. If you set this value to 0, no examples are stored. NOTE: The categorization_examples_limit applies only to analysis that uses categorization.

      • model_memory_limit number | string

    • background_persist_interval string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

    • custom_settings object

      Custom metadata about the job

    • daily_model_snapshot_retention_after_days number

      Advanced configuration option, which affects the automatic removal of old model snapshots for this job. It specifies a period of time (in days) after which only the first snapshot per day is retained. This period is relative to the timestamp of the most recent snapshot for this job.

    • data_description object Required
      Hide data_description attributes Show data_description attributes object
      • format string

        Only JSON format is supported at this time.

      • time_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • time_format string

        The time format, which can be epoch, epoch_ms, or a custom pattern. The value epoch refers to UNIX or Epoch time (the number of seconds since 1 Jan 1970). The value epoch_ms indicates that time is measured in milliseconds since the epoch. The epoch and epoch_ms time formats accept either integer or real values. Custom patterns must conform to the Java DateTimeFormatter class. When you use date-time formatting patterns, it is recommended that you provide the full date, time and time zone. For example: yyyy-MM-dd'T'HH:mm:ssX. If the pattern that you specify is not sufficient to produce a complete timestamp, job creation fails.

      • field_delimiter string
    • datafeed_config object
      Hide datafeed_config attributes Show datafeed_config attributes object
      • aggregations object

        If set, the datafeed performs aggregation searches. Support for aggregations is limited and should be used only with low cardinality data.

      • chunking_config object
        Hide chunking_config attributes Show chunking_config attributes object
        • mode string Required

          Values are auto, manual, or off.

        • time_span string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • datafeed_id string
      • delayed_data_check_config object
        Hide delayed_data_check_config attributes Show delayed_data_check_config attributes object
        • check_window string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • enabled boolean Required

          Specifies whether the datafeed periodically checks for delayed data.

      • frequency string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • indices string | array[string]
      • indices_options object

        Controls how to deal with unavailable concrete indices (closed or missing), how wildcard expressions are expanded to actual indices (all, closed or open indices) and how to deal with wildcard expressions that resolve to no indices.

        Hide indices_options attributes Show indices_options attributes object
        • allow_no_indices boolean

          If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

        • expand_wildcards string | array[string]
        • ignore_unavailable boolean

          If true, missing or closed indices are not included in the response.

        • ignore_throttled boolean

          If true, concrete, expanded or aliased indices are ignored when frozen.

      • job_id string
      • max_empty_searches number

        If a real-time datafeed has never seen any data (including during any initial training period) then it will automatically stop itself and close its associated job after this many real-time searches that return no documents. In other words, it will stop after frequency times max_empty_searches of real-time operation. If not set then a datafeed with no end time that sees no data will remain started until it is explicitly stopped.

      • query object

        An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

        External documentation
      • query_delay string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • runtime_mappings object
        Hide runtime_mappings attribute Show runtime_mappings attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • fields object

            For type composite

            Hide fields attribute Show fields attribute object
            • * object Additional properties
              Hide * attribute Show * attribute object
              • type string Required

                Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

          • fetch_fields array[object]

            For type lookup

            Hide fetch_fields attributes Show fetch_fields attributes object
            • field string Required

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • format string
          • format string

            A custom format for date type runtime fields.

          • input_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_index string
          • script object
            Hide script attributes Show script attributes object
            • source string

              The script source.

            • id string
            • params object

              Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

              Hide params attribute Show params attribute object
              • * object Additional properties
            • lang string

              Any of:

              Values are painless, expression, mustache, or java.

            • options object
              Hide options attribute Show options attribute object
              • * string Additional properties
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • script_fields object

        Specifies scripts that evaluate custom expressions and returns script fields to the datafeed. The detector configuration objects in a job can contain functions that use these script fields.

        Hide script_fields attribute Show script_fields attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • script object Required
            Hide script attributes Show script attributes object
            • source string

              The script source.

            • id string
            • params object

              Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

              Hide params attribute Show params attribute object
              • * object Additional properties
            • lang string

              Any of:

              Values are painless, expression, mustache, or java.

            • options object
              Hide options attribute Show options attribute object
              • * string Additional properties
          • ignore_failure boolean
      • scroll_size number

        The size parameter that is used in Elasticsearch searches when the datafeed does not use aggregations. The maximum value is the value of index.max_result_window, which is 10,000 by default.

    • description string

      A description of the job.

    • groups array[string]

      A list of job groups. A job can belong to no groups or many.

    • job_id string
    • job_type string

      Reserved for future use, currently set to anomaly_detector.

    • model_plot_config object
      Hide model_plot_config attributes Show model_plot_config attributes object
      • annotations_enabled boolean Generally available; Added in 7.9.0

        If true, enables calculation and storage of the model change annotations for each entity that is being analyzed.

      • enabled boolean

        If true, enables calculation and storage of the model bounds for each entity that is being analyzed.

      • terms string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • model_snapshot_retention_days number

      Advanced configuration option, which affects the automatic removal of old model snapshots for this job. It specifies the maximum period of time (in days) that snapshots are retained. This period is relative to the timestamp of the most recent snapshot for this job. The default value is 10, which means snapshots ten days older than the newest snapshot are deleted.

    • renormalization_window_days number

      Advanced configuration option. The period over which adjustments to the score are applied, as new data is seen. The default value is the longer of 30 days or 100 bucket_spans.

    • results_index_name string
    • results_retention_days number

      Advanced configuration option. The period of time (in days) that results are retained. Age is calculated relative to the timestamp of the latest bucket result. If this property has a non-null value, once per day at 00:30 (server time), results that are the specified number of days older than the latest bucket result are deleted from Elasticsearch. The default value is null, which means all results are retained. Annotations generated by the system also count as results for retention purposes; they are deleted after the same number of days as results. Annotations added by users are retained forever.

Responses

  • 200 application/json
POST /_ml/datafeeds/_preview
curl \
 --request POST 'http://api.example.com/_ml/datafeeds/_preview' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"datafeed_config":{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"datafeed_id":"string","delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0},"job_config":{"allow_lazy_open":true,"analysis_config":{"bucket_span":"string","":"string","categorization_field_name":"string","categorization_filters":["string"],"detectors":[{"by_field_name":"string","custom_rules":[{"actions":["skip_result"],"conditions":[{}],"scope":{}}],"detector_description":"string","detector_index":42.0,"exclude_frequent":"all","field_name":"string","function":"string","over_field_name":"string","partition_field_name":"string","use_null":true}],"influencers":["string"],"latency":"string","model_prune_window":"string","multivariate_by_fields":true,"per_partition_categorization":{"enabled":true,"stop_on_warn":true},"summary_count_field_name":"string"},"analysis_limits":{"categorization_examples_limit":42.0,"":42.0},"background_persist_interval":"string","custom_settings":{},"daily_model_snapshot_retention_after_days":42.0,"data_description":{"format":"string","time_field":"string","time_format":"string","field_delimiter":"string"},"datafeed_config":{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"datafeed_id":"string","delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0},"description":"string","groups":["string"],"job_id":"string","job_type":"string","model_plot_config":{"annotations_enabled":true,"enabled":true,"terms":"string"},"model_snapshot_retention_days":42.0,"renormalization_window_days":42.0,"results_index_name":"string","results_retention_days":42.0}}'
































Upgrade a snapshot Generally available; Added in 5.4.0

POST /_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_upgrade

Upgrade an anomaly detection model snapshot to the latest major version. Over time, older snapshot formats are deprecated and removed. Anomaly detection jobs support only snapshots that are from the current or previous major version. This API provides a means to upgrade a snapshot to the current major version. This aids in preparing the cluster for an upgrade to the next major version. Only one snapshot per anomaly detection job can be upgraded at a time and the upgraded snapshot cannot be the current snapshot of the anomaly detection job.

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

  • snapshot_id string Required

    A numerical character string that uniquely identifies the model snapshot.

Query parameters

  • wait_for_completion boolean

    When true, the API won’t respond until the upgrade is complete. Otherwise, it responds as soon as the upgrade task is assigned to a node.

  • timeout string

    Controls the time to wait for the request to complete.

    Values are -1 or 0.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • node string Required
    • completed boolean Required

      When true, this means the task is complete. When false, it is still running.

POST /_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_upgrade
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_upgrade' \
 --header "Authorization: $API_KEY"

Machine learning data frame analytics













Evaluate data frame analytics Generally available; Added in 7.3.0

POST /_ml/data_frame/_evaluate

The API packages together commonly used evaluation metrics for various types of machine learning features. This has been designed for use on indexes created by data frame analytics. Evaluation requires both a ground truth field and an analytics result field to be present.

Required authorization

  • Cluster privileges: monitor_ml
application/json

Body Required

  • evaluation object Required
    Hide evaluation attributes Show evaluation attributes object
    • classification object
      Hide classification attributes Show classification attributes object
      • actual_field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • predicted_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • top_classes_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • metrics object
        Hide metrics attributes Show metrics attributes object
        • auc_roc object
          Hide auc_roc attributes Show auc_roc attributes object
          • class_name string
          • include_curve boolean

            Whether or not the curve should be returned in addition to the score. Default value is false.

        • precision object

          Precision of predictions (per-class and average).

          Hide precision attribute Show precision attribute object
          • * object Additional properties
        • recall object

          Recall of predictions (per-class and average).

          Hide recall attribute Show recall attribute object
          • * object Additional properties
        • accuracy object

          Accuracy of predictions (per-class and overall).

          Hide accuracy attribute Show accuracy attribute object
          • * object Additional properties
        • multiclass_confusion_matrix object

          Multiclass confusion matrix.

          Hide multiclass_confusion_matrix attribute Show multiclass_confusion_matrix attribute object
          • * object Additional properties
    • outlier_detection object
      Hide outlier_detection attributes Show outlier_detection attributes object
      • actual_field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • predicted_probability_field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • metrics object
        Hide metrics attributes Show metrics attributes object
        • auc_roc object
          Hide auc_roc attributes Show auc_roc attributes object
          • class_name string
          • include_curve boolean

            Whether or not the curve should be returned in addition to the score. Default value is false.

        • precision object

          Precision of predictions (per-class and average).

          Hide precision attribute Show precision attribute object
          • * object Additional properties
        • recall object

          Recall of predictions (per-class and average).

          Hide recall attribute Show recall attribute object
          • * object Additional properties
        • confusion_matrix object

          Accuracy of predictions (per-class and overall).

          Hide confusion_matrix attribute Show confusion_matrix attribute object
          • * object Additional properties
    • regression object
      Hide regression attributes Show regression attributes object
      • actual_field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • predicted_field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • metrics object
        Hide metrics attributes Show metrics attributes object
        • mse object

          Average squared difference between the predicted values and the actual (ground truth) value. For more information, read this wiki article.

          Hide mse attribute Show mse attribute object
          • * object Additional properties
        • msle object
          Hide msle attribute Show msle attribute object
          • offset number

            Defines the transition point at which you switch from minimizing quadratic error to minimizing quadratic log error. Defaults to 1.

        • huber object
          Hide huber attribute Show huber attribute object
          • delta number

            Approximates 1/2 (prediction - actual)2 for values much less than delta and approximates a straight line with slope delta for values much larger than delta. Defaults to 1. Delta needs to be greater than 0.

        • r_squared object

          Proportion of the variance in the dependent variable that is predictable from the independent variables.

          Hide r_squared attribute Show r_squared attribute object
          • * object Additional properties
  • index string Required
  • query object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • classification object
      Hide classification attributes Show classification attributes object
      • auc_roc object
        Hide auc_roc attributes Show auc_roc attributes object
        • value number Required
        • curve array[object]
          Hide curve attributes Show curve attributes object
          • tpr number Required
          • fpr number Required
          • threshold number Required
      • accuracy object
        Hide accuracy attributes Show accuracy attributes object
        • classes array[object] Required
          Hide classes attributes Show classes attributes object
          • value number Required
          • class_name string Required
        • overall_accuracy number Required
      • multiclass_confusion_matrix object
        Hide multiclass_confusion_matrix attributes Show multiclass_confusion_matrix attributes object
        • confusion_matrix array[object] Required
          Hide confusion_matrix attributes Show confusion_matrix attributes object
          • actual_class string Required
          • actual_class_doc_count number Required
          • predicted_classes array[object] Required
          • other_predicted_class_doc_count number Required
        • other_actual_class_count number Required
      • precision object
        Hide precision attributes Show precision attributes object
        • classes array[object] Required
          Hide classes attributes Show classes attributes object
          • value number Required
          • class_name string Required
        • avg_precision number Required
      • recall object
        Hide recall attributes Show recall attributes object
        • classes array[object] Required
          Hide classes attributes Show classes attributes object
          • value number Required
          • class_name string Required
        • avg_recall number Required
    • outlier_detection object
      Hide outlier_detection attributes Show outlier_detection attributes object
      • auc_roc object
        Hide auc_roc attributes Show auc_roc attributes object
        • value number Required
        • curve array[object]
          Hide curve attributes Show curve attributes object
          • tpr number Required
          • fpr number Required
          • threshold number Required
      • precision object

        Set the different thresholds of the outlier score at where the metric is calculated.

        Hide precision attribute Show precision attribute object
        • * number Additional properties
      • recall object

        Set the different thresholds of the outlier score at where the metric is calculated.

        Hide recall attribute Show recall attribute object
        • * number Additional properties
      • confusion_matrix object

        Set the different thresholds of the outlier score at where the metrics (tp - true positive, fp - false positive, tn - true negative, fn - false negative) are calculated.

        Hide confusion_matrix attribute Show confusion_matrix attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • tp number Required

            True Positive

          • fp number Required

            False Positive

          • tn number Required

            True Negative

          • fn number Required

            False Negative

    • regression object
      Hide regression attributes Show regression attributes object
      • huber object
        Hide huber attribute Show huber attribute object
        • value number Required
      • mse object
        Hide mse attribute Show mse attribute object
        • value number Required
      • msle object
        Hide msle attribute Show msle attribute object
        • value number Required
      • r_squared object
        Hide r_squared attribute Show r_squared attribute object
        • value number Required
POST /_ml/data_frame/_evaluate
curl \
 --request POST 'http://api.example.com/_ml/data_frame/_evaluate' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"index\": \"animal_classification\",\n  \"evaluation\": {\n    \"classification\": {\n      \"actual_field\": \"animal_class\",\n      \"predicted_field\": \"ml.animal_class_prediction\",\n      \"metrics\": {\n        \"multiclass_confusion_matrix\": {}\n      }\n    }\n  }\n}"'
Run `POST _ml/data_frame/_evaluate` to evaluate a a classification job for an annotated index. The `actual_field` contains the ground truth for classification. The `predicted_field` contains the predicted value calculated by the classification analysis.
{
  "index": "animal_classification",
  "evaluation": {
    "classification": {
      "actual_field": "animal_class",
      "predicted_field": "ml.animal_class_prediction",
      "metrics": {
        "multiclass_confusion_matrix": {}
      }
    }
  }
}
Run `POST _ml/data_frame/_evaluate` to evaluate a classification job with AUC ROC metrics for an annotated index. The `actual_field` contains the ground truth value for the actual animal classification. This is required in order to evaluate results. The `class_name` specifies the class name that is treated as positive during the evaluation, all the other classes are treated as negative.
{
  "index": "animal_classification",
  "evaluation": {
    "classification": {
      "actual_field": "animal_class",
      "metrics": {
        "auc_roc": {
          "class_name": "dog"
        }
      }
    }
  }
}
Run `POST _ml/data_frame/_evaluate` to evaluate an outlier detection job for an annotated index.
{
  "index": "my_analytics_dest_index",
  "evaluation": {
    "outlier_detection": {
      "actual_field": "is_outlier",
      "predicted_probability_field": "ml.outlier_score"
    }
  }
}
Run `POST _ml/data_frame/_evaluate` to evaluate the testing error of a regression job for an annotated index. The term query in the body limits evaluation to be performed on the test split only. The `actual_field` contains the ground truth for house prices. The `predicted_field` contains the house price calculated by the regression analysis.
{
  "index": "house_price_predictions",
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "ml.is_training": false
          }
        }
      ]
    }
  },
  "evaluation": {
    "regression": {
      "actual_field": "price",
      "predicted_field": "ml.price_prediction",
      "metrics": {
        "r_squared": {},
        "mse": {},
        "msle": {
          "offset": 10
        },
        "huber": {
          "delta": 1.5
        }
      }
    }
  }
}
Run `POST _ml/data_frame/_evaluate` to evaluate the training error of a regression job for an annotated index. The term query in the body limits evaluation to be performed on the training split only. The `actual_field` contains the ground truth for house prices. The `predicted_field` contains the house price calculated by the regression analysis.
{
  "index": "house_price_predictions",
  "query": {
    "term": {
      "ml.is_training": {
        "value": true
      }
    }
  },
  "evaluation": {
    "regression": {
      "actual_field": "price",
      "predicted_field": "ml.price_prediction",
      "metrics": {
        "r_squared": {},
        "mse": {},
        "msle": {},
        "huber": {}
      }
    }
  }
}
A succesful response from `POST _ml/data_frame/_evaluate` to evaluate a classification analysis job for an annotated index. The `actual_class` contains the name of the class the analysis tried to predict. The `actual_class_doc_count` is the number of documents in the index belonging to the `actual_class`. The `predicted_classes` object contains the list of the predicted classes and the number of predictions associated with the class.
{
  "classification": {
    "multiclass_confusion_matrix": {
      "confusion_matrix": [
        {
          "actual_class": "cat",
          "actual_class_doc_count": 12,
          "predicted_classes": [
            {
              "predicted_class": "cat",
              "count": 12
            },
            {
              "predicted_class": "dog",
              "count": 0
            }
          ],
          "other_predicted_class_doc_count": 0
        },
        {
          "actual_class": "dog",
          "actual_class_doc_count": 11,
          "predicted_classes": [
            {
              "predicted_class": "dog",
              "count": 7
            },
            {
              "predicted_class": "cat",
              "count": 4
            }
          ],
          "other_predicted_class_doc_count": 0
        }
      ],
      "other_actual_class_count": 0
    }
  }
}
A succesful response from `POST _ml/data_frame/_evaluate` to evaluate a classification analysis job with the AUC ROC metrics for an annotated index.
{
  "classification": {
    "auc_roc": {
      "value": 0.8941788639536681
    }
  }
}
A successful response from `POST _ml/data_frame/_evaluate` to evaluate an outlier detection job.
{
  "outlier_detection": {
    "auc_roc": {
      "value": 0.9258475774641445
    },
    "confusion_matrix": {
      "0.25": {
        "tp": 5,
        "fp": 9,
        "tn": 204,
        "fn": 5
      },
      "0.5": {
        "tp": 1,
        "fp": 5,
        "tn": 208,
        "fn": 9
      },
      "0.75": {
        "tp": 0,
        "fp": 4,
        "tn": 209,
        "fn": 10
      }
    },
    "precision": {
      "0.25": 0.35714285714285715,
      "0.5": 0.16666666666666666,
      "0.75": 0
    },
    "recall": {
      "0.25": 0.5,
      "0.5": 0.1,
      "0.75": 0
    }
  }
}








Explain data frame analytics config Generally available; Added in 7.3.0

GET /_ml/data_frame/analytics/{id}/_explain

This API provides explanations for a data frame analytics config that either exists already or one that has not been created yet. The following explanations are provided:

  • which fields are included or not in the analysis and why,
  • how much memory is estimated to be required. The estimate can be used when deciding the appropriate value for model_memory_limit setting later on. If you have object fields or fields that are excluded via source filtering, they are not included in the explanation.

Required authorization

  • Cluster privileges: monitor_ml

Path parameters

  • id string Required

    Identifier for the data frame analytics job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.

application/json

Body

  • source object
    Hide source attributes Show source attributes object
    • index string | array[string] Required
    • runtime_mappings object
      Hide runtime_mappings attribute Show runtime_mappings attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • format string
        • format string

          A custom format for date type runtime fields.

        • input_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_index string
        • script object
          Hide script attributes Show script attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties
          • lang string

            Any of:

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • type string Required

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • _source object
      Hide _source attributes Show _source attributes object
      • includes array[string] Required

        An array of strings that defines the fields that will be excluded from the analysis. You do not need to add fields with unsupported data types to excludes, these fields are excluded from the analysis automatically.

      • excludes array[string] Required

        An array of strings that defines the fields that will be included in the analysis.

    • query object

      The Elasticsearch query domain-specific language (DSL). This value corresponds to the query object in an Elasticsearch search POST body. All the options that are supported by Elasticsearch can be used, as this object is passed verbatim to Elasticsearch. By default, this property has the following value: {"match_all": {}}.

      Query DSL
  • dest object
    Hide dest attributes Show dest attributes object
    • index string Required
    • results_field string

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • analysis object
    Hide analysis attributes Show analysis attributes object
    • classification object
      Hide classification attributes Show classification attributes object
      • alpha number

        Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.

      • dependent_variable string Required

        Defines which field of the document is to be predicted. It must match one of the fields in the index being used to train. If this field is missing from a document, then that document will not be used for training, but a prediction with the trained model will be generated for it. It is also known as continuous target variable. For classification analysis, the data type of the field must be numeric (integer, short, long, byte), categorical (ip or keyword), or boolean. There must be no more than 30 different values in this field. For regression analysis, the data type of the field must be numeric.

      • downsample_factor number

        Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.

      • early_stopping_enabled boolean

        Advanced configuration option. Specifies whether the training process should finish if it is not finding any better performing models. If disabled, the training process can take significantly longer and the chance of finding a better performing model is unremarkable.

      • eta number

        Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.

      • eta_growth_rate_per_tree number

        Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.

      • feature_bag_fraction number

        Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.

      • feature_processors array[object]

        Advanced configuration option. A collection of feature preprocessors that modify one or more included fields. The analysis uses the resulting one or more features instead of the original document field. However, these features are ephemeral; they are not stored in the destination index. Multiple feature_processors entries can refer to the same document fields. Automatic categorical feature encoding still occurs for the fields that are unprocessed by a custom processor or that have categorical values. Use this property only if you want to override the automatic feature encoding of the specified fields.

        Hide feature_processors attributes Show feature_processors attributes object
        • frequency_encoding object
          Hide frequency_encoding attributes Show frequency_encoding attributes object
          • feature_name string Required
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • frequency_map object Required

            The resulting frequency map for the field value. If the field value is missing from the frequency_map, the resulting value is 0.

        • multi_encoding object
          Hide multi_encoding attribute Show multi_encoding attribute object
          • processors array[number] Required

            The ordered array of custom processors to execute. Must be more than 1.

        • n_gram_encoding object
          Hide n_gram_encoding attributes Show n_gram_encoding attributes object
          • feature_prefix string

            The feature name prefix. Defaults to ngram__.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • length number

            Specifies the length of the n-gram substring. Defaults to 50. Must be greater than 0.

          • n_grams array[number] Required

            Specifies which n-grams to gather. It’s an array of integer values where the minimum value is 1, and a maximum value is 5.

          • start number

            Specifies the zero-indexed start of the n-gram substring. Negative values are allowed for encoding n-grams of string suffixes. Defaults to 0.

          • custom boolean
        • one_hot_encoding object
          Hide one_hot_encoding attributes Show one_hot_encoding attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • hot_map string Required

            The one hot map mapping the field value with the column name.

        • target_mean_encoding object
          Hide target_mean_encoding attributes Show target_mean_encoding attributes object
          • default_value number Required

            The default value if field value is not found in the target_map.

          • feature_name string Required
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_map object Required

            The field value to target mean transition map.

      • gamma number

        Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

      • lambda number

        Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

      • max_optimization_rounds_per_hyperparameter number

        Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.

      • max_trees number

        Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.

      • num_top_feature_importance_values number

        Advanced configuration option. Specifies the maximum number of feature importance values per document to return. By default, no feature importance calculation occurs.

      • prediction_field_name string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • randomize_seed number

        Defines the seed for the random generator that is used to pick training data. By default, it is randomly generated. Set it to a specific value to use the same training data each time you start a job (assuming other related parameters such as source and analyzed_fields are the same).

      • soft_tree_depth_limit number

        Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.

      • soft_tree_depth_tolerance number

        Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.

      • training_percent string | number

      • class_assignment_objective string
      • num_top_classes number

        Defines the number of categories for which the predicted probabilities are reported. It must be non-negative or -1. If it is -1 or greater than the total number of categories, probabilities are reported for all categories; if you have a large number of categories, there could be a significant effect on the size of your destination index. NOTE: To use the AUC ROC evaluation method, num_top_classes must be set to -1 or a value greater than or equal to the total number of categories.

    • outlier_detection object
      Hide outlier_detection attributes Show outlier_detection attributes object
      • compute_feature_influence boolean

        Specifies whether the feature influence calculation is enabled.

      • feature_influence_threshold number

        The minimum outlier score that a document needs to have in order to calculate its feature influence score. Value range: 0-1.

      • method string

        The method that outlier detection uses. Available methods are lof, ldof, distance_kth_nn, distance_knn, and ensemble. The default value is ensemble, which means that outlier detection uses an ensemble of different methods and normalises and combines their individual outlier scores to obtain the overall outlier score.

      • n_neighbors number

        Defines the value for how many nearest neighbors each method of outlier detection uses to calculate its outlier score. When the value is not set, different values are used for different ensemble members. This default behavior helps improve the diversity in the ensemble; only override it if you are confident that the value you choose is appropriate for the data set.

      • outlier_fraction number

        The proportion of the data set that is assumed to be outlying prior to outlier detection. For example, 0.05 means it is assumed that 5% of values are real outliers and 95% are inliers.

      • standardization_enabled boolean

        If true, the following operation is performed on the columns before computing outlier scores: (x_i - mean(x_i)) / sd(x_i).

    • regression object
      Hide regression attributes Show regression attributes object
      • alpha number

        Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.

      • dependent_variable string Required

        Defines which field of the document is to be predicted. It must match one of the fields in the index being used to train. If this field is missing from a document, then that document will not be used for training, but a prediction with the trained model will be generated for it. It is also known as continuous target variable. For classification analysis, the data type of the field must be numeric (integer, short, long, byte), categorical (ip or keyword), or boolean. There must be no more than 30 different values in this field. For regression analysis, the data type of the field must be numeric.

      • downsample_factor number

        Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.

      • early_stopping_enabled boolean

        Advanced configuration option. Specifies whether the training process should finish if it is not finding any better performing models. If disabled, the training process can take significantly longer and the chance of finding a better performing model is unremarkable.

      • eta number

        Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.

      • eta_growth_rate_per_tree number

        Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.

      • feature_bag_fraction number

        Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.

      • feature_processors array[object]

        Advanced configuration option. A collection of feature preprocessors that modify one or more included fields. The analysis uses the resulting one or more features instead of the original document field. However, these features are ephemeral; they are not stored in the destination index. Multiple feature_processors entries can refer to the same document fields. Automatic categorical feature encoding still occurs for the fields that are unprocessed by a custom processor or that have categorical values. Use this property only if you want to override the automatic feature encoding of the specified fields.

        Hide feature_processors attributes Show feature_processors attributes object
        • frequency_encoding object
          Hide frequency_encoding attributes Show frequency_encoding attributes object
          • feature_name string Required
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • frequency_map object Required

            The resulting frequency map for the field value. If the field value is missing from the frequency_map, the resulting value is 0.

        • multi_encoding object
          Hide multi_encoding attribute Show multi_encoding attribute object
          • processors array[number] Required

            The ordered array of custom processors to execute. Must be more than 1.

        • n_gram_encoding object
          Hide n_gram_encoding attributes Show n_gram_encoding attributes object
          • feature_prefix string

            The feature name prefix. Defaults to ngram__.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • length number

            Specifies the length of the n-gram substring. Defaults to 50. Must be greater than 0.

          • n_grams array[number] Required

            Specifies which n-grams to gather. It’s an array of integer values where the minimum value is 1, and a maximum value is 5.

          • start number

            Specifies the zero-indexed start of the n-gram substring. Negative values are allowed for encoding n-grams of string suffixes. Defaults to 0.

          • custom boolean
        • one_hot_encoding object
          Hide one_hot_encoding attributes Show one_hot_encoding attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • hot_map string Required

            The one hot map mapping the field value with the column name.

        • target_mean_encoding object
          Hide target_mean_encoding attributes Show target_mean_encoding attributes object
          • default_value number Required

            The default value if field value is not found in the target_map.

          • feature_name string Required
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_map object Required

            The field value to target mean transition map.

      • gamma number

        Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

      • lambda number

        Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

      • max_optimization_rounds_per_hyperparameter number

        Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.

      • max_trees number

        Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.

      • num_top_feature_importance_values number

        Advanced configuration option. Specifies the maximum number of feature importance values per document to return. By default, no feature importance calculation occurs.

      • prediction_field_name string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • randomize_seed number

        Defines the seed for the random generator that is used to pick training data. By default, it is randomly generated. Set it to a specific value to use the same training data each time you start a job (assuming other related parameters such as source and analyzed_fields are the same).

      • soft_tree_depth_limit number

        Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.

      • soft_tree_depth_tolerance number

        Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.

      • training_percent string | number

      • loss_function string

        The loss function used during regression. Available options are mse (mean squared error), msle (mean squared logarithmic error), huber (Pseudo-Huber loss).

      • loss_function_parameter number

        A positive number that is used as a parameter to the loss_function.

  • description string

    A description of the job.

  • model_memory_limit string

    The approximate maximum amount of memory resources that are permitted for analytical processing. If your elasticsearch.yml file contains an xpack.ml.max_model_memory_limit setting, an error occurs when you try to create data frame analytics jobs that have model_memory_limit values greater than that setting.

  • max_num_threads number

    The maximum number of threads to be used by the analysis. Using more threads may decrease the time necessary to complete the analysis at the cost of using more CPU. Note that the process may use additional threads for operational functionality other than the analysis itself.

  • analyzed_fields object
    Hide analyzed_fields attributes Show analyzed_fields attributes object
    • includes array[string] Required

      An array of strings that defines the fields that will be excluded from the analysis. You do not need to add fields with unsupported data types to excludes, these fields are excluded from the analysis automatically.

    • excludes array[string] Required

      An array of strings that defines the fields that will be included in the analysis.

  • allow_lazy_start boolean

    Specifies whether this job can start when there is insufficient machine learning node capacity for it to be immediately assigned to a node.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • field_selection array[object] Required

      An array of objects that explain selection for each field, sorted by the field names.

      Hide field_selection attributes Show field_selection attributes object
      • is_included boolean Required

        Whether the field is selected to be included in the analysis.

      • is_required boolean Required

        Whether the field is required.

      • feature_type string

        The feature type of this field for the analysis. May be categorical or numerical.

      • mapping_types array[string] Required

        The mapping types of the field.

      • name string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • reason string

        The reason a field is not selected to be included in the analysis.

    • memory_estimation object Required
      Hide memory_estimation attributes Show memory_estimation attributes object
      • expected_memory_with_disk string Required

        Estimated memory usage under the assumption that overflowing to disk is allowed during data frame analytics. expected_memory_with_disk is usually smaller than expected_memory_without_disk as using disk allows to limit the main memory needed to perform data frame analytics.

      • expected_memory_without_disk string Required

        Estimated memory usage under the assumption that the whole data frame analytics should happen in memory (i.e. without overflowing to disk).

GET /_ml/data_frame/analytics/{id}/_explain
curl \
 --request GET 'http://api.example.com/_ml/data_frame/analytics/{id}/_explain' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"source\": {\n    \"index\": \"houses_sold_last_10_yrs\"\n  },\n  \"analysis\": {\n    \"regression\": {\n      \"dependent_variable\": \"price\"\n    }\n  }\n}"'
Request example
Run `POST _ml/data_frame/analytics/_explain` to explain a data frame analytics job configuration.
{
  "source": {
    "index": "houses_sold_last_10_yrs"
  },
  "analysis": {
    "regression": {
      "dependent_variable": "price"
    }
  }
}
Response examples (200)
A succesful response for explaining a data frame analytics job configuration.
{
  "field_selection": [
    {
      "field": "number_of_bedrooms",
      "mappings_types": [
        "integer"
      ],
      "is_included": true,
      "is_required": false,
      "feature_type": "numerical"
    },
    {
      "field": "postcode",
      "mappings_types": [
        "text"
      ],
      "is_included": false,
      "is_required": false,
      "reason": "[postcode.keyword] is preferred because it is aggregatable"
    },
    {
      "field": "postcode.keyword",
      "mappings_types": [
        "keyword"
      ],
      "is_included": true,
      "is_required": false,
      "feature_type": "categorical"
    },
    {
      "field": "price",
      "mappings_types": [
        "float"
      ],
      "is_included": true,
      "is_required": true,
      "feature_type": "numerical"
    }
  ],
  "memory_estimation": {
    "expected_memory_without_disk": "128MB",
    "expected_memory_with_disk": "32MB"
  }
}
















































Get trained model configuration info Generally available; Added in 7.10.0

GET /_ml/trained_models/{model_id}

Required authorization

  • Cluster privileges: monitor_ml

Path parameters

  • model_id string | array[string] Required

    The unique identifier of the trained model or a model alias.

    You can get information for multiple trained models in a single API request by using a comma-separated list of model IDs or a wildcard expression.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    • Contains wildcard expressions and there are no models that match.
    • Contains the _all string or no identifiers and there are no matches.
    • Contains wildcard expressions and there are only partial matches.

    If true, it returns an empty array when there are no matches and the subset of results when there are partial matches.

  • decompress_definition boolean

    Specifies whether the included model definition should be returned as a JSON map (true) or in a custom compressed format (false).

  • exclude_generated boolean

    Indicates if certain fields should be removed from the configuration on retrieval. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.

  • from number

    Skips the specified number of models.

  • include string

    A comma delimited string of optional fields to include in the response body.

    Values are definition, feature_importance_baseline, hyperparameters, total_feature_importance, or definition_status.

  • include_model_definition boolean Deprecated

    parameter is deprecated! Use [include=definition] instead

  • size number

    Specifies the maximum number of models to obtain.

  • tags string | array[string]

    A comma delimited string of tags. A trained model can have many tags, or none. When supplied, only trained models that contain all the supplied tags are returned.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • trained_model_configs array[object] Required

      An array of trained model resources, which are sorted by the model_id value in ascending order.

      Hide trained_model_configs attributes Show trained_model_configs attributes object
      • model_id string Required
      • model_type string

        Values are tree_ensemble, lang_ident, or pytorch.

      • tags array[string] Required

        A comma delimited string of tags. A trained model can have many tags, or none.

      • version string
      • compressed_definition string
      • created_by string

        Information on the creator of the trained model.

      • create_time string | number

        A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

        One of:
      • default_field_map object

        Any field map described in the inference configuration takes precedence.

        Hide default_field_map attribute Show default_field_map attribute object
        • * string Additional properties
      • description string

        The free-text description of the trained model.

      • estimated_heap_memory_usage_bytes number

        The estimated heap usage in bytes to keep the trained model in memory.

      • estimated_operations number

        The estimated number of operations to use the trained model.

      • fully_defined boolean

        True if the full model definition is present.

      • inference_config object

        Inference configuration provided when storing the model config

        Hide inference_config attributes Show inference_config attributes object
        • regression object
          Hide regression attributes Show regression attributes object
          • results_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • num_top_feature_importance_values number

            Specifies the maximum number of feature importance values per document.

        • classification object
          Hide classification attributes Show classification attributes object
          • num_top_classes number

            Specifies the number of top class predictions to return. Defaults to 0.

          • num_top_feature_importance_values number

            Specifies the maximum number of feature importance values per document.

          • prediction_field_type string

            Specifies the type of the predicted field to write. Acceptable values are: string, number, boolean. When boolean is provided 1.0 is transformed to true and 0.0 to false.

          • results_field string

            The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

          • top_classes_results_field string

            Specifies the field to which the top classes are written. Defaults to top_classes.

        • text_classification object

          Text classification configuration options

          Hide text_classification attributes Show text_classification attributes object
          • num_top_classes number

            Specifies the number of top class predictions to return. Defaults to 0.

          • tokenization object

            Tokenization options stored in inference configuration

            Hide tokenization attributes Show tokenization attributes object
            • bert
            • bert_ja
            • mpnet
            • roberta
            • xlm_roberta
          • results_field string

            The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

          • classification_labels array[string]

            Classification labels to apply other than the stored labels. Must have the same deminsions as the default configured labels

          • vocabulary object
            Hide vocabulary attribute Show vocabulary attribute object
            • index string Required
        • zero_shot_classification object

          Zero shot classification configuration options

          Hide zero_shot_classification attributes Show zero_shot_classification attributes object
          • tokenization object

            Tokenization options stored in inference configuration

            Hide tokenization attributes Show tokenization attributes object
            • bert
            • bert_ja
            • mpnet
            • roberta
            • xlm_roberta
          • hypothesis_template string

            Hypothesis template used when tokenizing labels for prediction

          • classification_labels array[string] Required

            The zero shot classification labels indicating entailment, neutral, and contradiction Must contain exactly and only entailment, neutral, and contradiction

          • results_field string

            The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

          • multi_label boolean

            Indicates if more than one true label exists.

          • labels array[string]

            The labels to predict.

        • fill_mask object

          Fill mask inference options

          Hide fill_mask attributes Show fill_mask attributes object
          • mask_token string

            The string/token which will be removed from incoming documents and replaced with the inference prediction(s). In a response, this field contains the mask token for the specified model/tokenizer. Each model and tokenizer has a predefined mask token which cannot be changed. Thus, it is recommended not to set this value in requests. However, if this field is present in a request, its value must match the predefined value for that model/tokenizer, otherwise the request will fail.

          • num_top_classes number

            Specifies the number of top class predictions to return. Defaults to 0.

          • tokenization object

            Tokenization options stored in inference configuration

            Hide tokenization attributes Show tokenization attributes object
            • bert
            • bert_ja
            • mpnet
            • roberta
            • xlm_roberta
          • results_field string

            The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

          • vocabulary object Required
            Hide vocabulary attribute Show vocabulary attribute object
            • index string Required
        • learning_to_rank object
          Hide learning_to_rank attributes Show learning_to_rank attributes object
          • default_params object
            Hide default_params attribute Show default_params attribute object
            • * object Additional properties
          • feature_extractors array[object]
          • num_top_feature_importance_values number Required
        • ner object

          Named entity recognition options

          Hide ner attributes Show ner attributes object
          • tokenization object

            Tokenization options stored in inference configuration

            Hide tokenization attributes Show tokenization attributes object
            • bert
            • bert_ja
            • mpnet
            • roberta
            • xlm_roberta
          • results_field string

            The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

          • classification_labels array[string]

            The token classification labels. Must be IOB formatted tags

          • vocabulary object
            Hide vocabulary attribute Show vocabulary attribute object
            • index string Required
        • pass_through object

          Pass through configuration options

          Hide pass_through attributes Show pass_through attributes object
          • tokenization object

            Tokenization options stored in inference configuration

            Hide tokenization attributes Show tokenization attributes object
            • bert
            • bert_ja
            • mpnet
            • roberta
            • xlm_roberta
          • results_field string

            The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

          • vocabulary object
            Hide vocabulary attribute Show vocabulary attribute object
            • index string Required
        • text_embedding object

          Text embedding inference options

          Hide text_embedding attributes Show text_embedding attributes object
          • embedding_size number

            The number of dimensions in the embedding output

          • tokenization object

            Tokenization options stored in inference configuration

            Hide tokenization attributes Show tokenization attributes object
            • bert
            • bert_ja
            • mpnet
            • roberta
            • xlm_roberta
          • results_field string

            The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

          • vocabulary object Required
            Hide vocabulary attribute Show vocabulary attribute object
            • index string Required
        • text_expansion object

          Text expansion inference options

          Hide text_expansion attributes Show text_expansion attributes object
          • tokenization object

            Tokenization options stored in inference configuration

            Hide tokenization attributes Show tokenization attributes object
            • bert
            • bert_ja
            • mpnet
            • roberta
            • xlm_roberta
          • results_field string

            The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

          • vocabulary object Required
            Hide vocabulary attribute Show vocabulary attribute object
            • index string Required
        • question_answering object

          Question answering inference options

          Hide question_answering attributes Show question_answering attributes object
          • num_top_classes number

            Specifies the number of top class predictions to return. Defaults to 0.

          • tokenization object

            Tokenization options stored in inference configuration

            Hide tokenization attributes Show tokenization attributes object
            • bert
            • bert_ja
            • mpnet
            • roberta
            • xlm_roberta
          • results_field string

            The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

          • max_answer_length number

            The maximum answer length to consider

      • input object Required
        Hide input attribute Show input attribute object
        • field_names array[string] Required

          An array of input field names for the model.

      • license_level string

        The license level of the trained model.

      • metadata object
        Hide metadata attributes Show metadata attributes object
        • model_aliases array[string]
        • feature_importance_baseline object

          An object that contains the baseline for feature importance values. For regression analysis, it is a single value. For classification analysis, there is a value for each class.

          Hide feature_importance_baseline attribute Show feature_importance_baseline attribute object
          • * string Additional properties
        • hyperparameters array[object]

          List of the available hyperparameters optimized during the fine_parameter_tuning phase as well as specified by the user.

          Hide hyperparameters attributes Show hyperparameters attributes object
          • absolute_importance number

            A positive number showing how much the parameter influences the variation of the loss function. For hyperparameters with values that are not specified by the user but tuned during hyperparameter optimization.

          • name string Required
          • relative_importance number

            A number between 0 and 1 showing the proportion of influence on the variation of the loss function among all tuned hyperparameters. For hyperparameters with values that are not specified by the user but tuned during hyperparameter optimization.

          • supplied boolean Required

            Indicates if the hyperparameter is specified by the user (true) or optimized (false).

          • value number Required

            The value of the hyperparameter, either optimized or specified by the user.

        • total_feature_importance array[object]

          An array of the total feature importance for each feature used from the training data set. This array of objects is returned if data frame analytics trained the model and the request includes total_feature_importance in the include request parameter.

          Hide total_feature_importance attributes Show total_feature_importance attributes object
          • feature_name string Required
          • importance array[object] Required

            A collection of feature importance statistics related to the training data set for this particular feature.

          • classes array[object] Required

            If the trained model is a classification model, feature importance statistics are gathered per target class value.

      • model_size_bytes number | string

      • model_package object
        Hide model_package attributes Show model_package attributes object
        • create_time number

          Time unit for milliseconds

        • description string
        • inference_config object
          Hide inference_config attribute Show inference_config attribute object
          • * object Additional properties
        • metadata object
          Hide metadata attribute Show metadata attribute object
          • * object Additional properties
        • minimum_version string
        • model_repository string
        • model_type string
        • packaged_model_id string Required
        • platform_architecture string
        • prefix_strings object
          Hide prefix_strings attributes Show prefix_strings attributes object
          • ingest string

            String prepended to input at ingest

        • size number | string

        • sha256 string
        • tags array[string]
        • vocabulary_file string
      • location object
        Hide location attribute Show location attribute object
        • index object Required
          Hide index attribute Show index attribute object
          • name string Required
      • platform_architecture string
      • prefix_strings object
        Hide prefix_strings attributes Show prefix_strings attributes object
        • ingest string

          String prepended to input at ingest

GET /_ml/trained_models/{model_id}
curl \
 --request GET 'http://api.example.com/_ml/trained_models/{model_id}' \
 --header "Authorization: $API_KEY"








Create or update a trained model alias Generally available; Added in 7.13.0

PUT /_ml/trained_models/{model_id}/model_aliases/{model_alias}

A trained model alias is a logical name used to reference a single trained model. You can use aliases instead of trained model identifiers to make it easier to reference your models. For example, you can use aliases in inference aggregations and processors. An alias must be unique and refer to only a single trained model. However, you can have multiple aliases for each trained model. If you use this API to update an alias such that it references a different trained model ID and the model uses a different type of data frame analytics, an error occurs. For example, this situation occurs if you have a trained model for regression analysis and a trained model for classification analysis; you cannot reassign an alias from one type of trained model to another. If you use this API to update an alias and there are very few input fields in common between the old and new trained models for the model alias, the API returns a warning.

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • model_id string Required

    The identifier for the trained model that the alias refers to.

  • model_alias string Required

    The alias to create or update. This value cannot end in numbers.

Query parameters

  • reassign boolean

    Specifies whether the alias gets reassigned to the specified trained model if it is already assigned to a different model. If the alias is already assigned and this parameter is false, the API returns an error.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_ml/trained_models/{model_id}/model_aliases/{model_alias}
curl \
 --request PUT 'http://api.example.com/_ml/trained_models/{model_id}/model_aliases/{model_alias}' \
 --header "Authorization: $API_KEY"




























Create a trained model vocabulary Generally available; Added in 8.0.0

PUT /_ml/trained_models/{model_id}/vocabulary

This API is supported only for natural language processing (NLP) models. The vocabulary is stored in the index as described in inference_config.*.vocabulary of the trained model definition.

Required authorization

  • Cluster privileges: manage_ml

Path parameters

  • model_id string Required

    The unique identifier of the trained model.

application/json

Body Required

  • vocabulary array[string] Required

    The model vocabulary, which must not be empty.

  • merges array[string] Generally available; Added in 8.2.0

    The optional model merges if required by the tokenizer.

  • scores array[number] Generally available; Added in 8.9.0

    The optional vocabulary value scores if required by the tokenizer.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_ml/trained_models/{model_id}/vocabulary
curl \
 --request PUT 'http://api.example.com/_ml/trained_models/{model_id}/vocabulary' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"vocabulary\": [\n    \"[PAD]\",\n    \"[unused0]\",\n  ]\n}"'
Request example
An example body for a `PUT _ml/trained_models/elastic__distilbert-base-uncased-finetuned-conll03-english/vocabulary` request.
{
  "vocabulary": [
    "[PAD]",
    "[unused0]",
  ]
}












Migration





Create an index from a source index Technical preview; Added in 8.18.0

PUT /_create_from/{source}/{dest}

Copy the mappings and settings from the source index to a destination index while allowing request settings and mappings to override the source values.

Path parameters

  • source string Required

    The source index or data stream name

  • dest string Required

    The destination index or data stream name

application/json

Body Required

  • mappings_override object
    Hide mappings_override attributes Show mappings_override attributes object
    • all_field object
      Hide all_field attributes Show all_field attributes object
      • analyzer string Required
      • enabled boolean Required
      • omit_norms boolean Required
      • search_analyzer string Required
      • similarity string Required
      • store boolean Required
      • store_term_vector_offsets boolean Required
      • store_term_vector_payloads boolean Required
      • store_term_vector_positions boolean Required
      • store_term_vectors boolean Required
    • date_detection boolean
    • dynamic string

      Values are strict, runtime, true, or false.

    • dynamic_date_formats array[string]
    • dynamic_templates array[object]
    • _field_names object
      Hide _field_names attribute Show _field_names attribute object
      • enabled boolean Required
    • index_field object
      Hide index_field attribute Show index_field attribute object
      • enabled boolean Required
    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties
    • numeric_detection boolean
    • properties object
    • _routing object
      Hide _routing attribute Show _routing attribute object
      • required boolean Required
    • _size object
      Hide _size attribute Show _size attribute object
      • enabled boolean Required
    • _source object
      Hide _source attributes Show _source attributes object
      • compress boolean
      • compress_threshold string
      • enabled boolean
      • excludes array[string]
      • includes array[string]
      • mode string

        Values are disabled, stored, or synthetic.

    • runtime object
      Hide runtime attribute Show runtime attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • format string
        • format string

          A custom format for date type runtime fields.

        • input_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_index string
        • script object
          Hide script attributes Show script attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties
          • lang string

            Any of:

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • type string Required

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • enabled boolean
    • subobjects string

      Values are true or false.

    • _data_stream_timestamp object
      Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
      • enabled boolean Required
  • settings_override object
    Index settings
  • remove_index_blocks boolean

    If index blocks should be removed when creating destination index (optional)

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required
    • index string Required
    • shards_acknowledged boolean Required
PUT /_create_from/{source}/{dest}
curl \
 --request PUT 'http://api.example.com/_create_from/{source}/{dest}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"mappings_override":{"all_field":{"analyzer":"string","enabled":true,"omit_norms":true,"search_analyzer":"string","similarity":"string","store":true,"store_term_vector_offsets":true,"store_term_vector_payloads":true,"store_term_vector_positions":true,"store_term_vectors":true},"date_detection":true,"dynamic":"strict","dynamic_date_formats":["string"],"dynamic_templates":[{}],"_field_names":{"enabled":true},"index_field":{"enabled":true},"_meta":{"additionalProperty1":{},"additionalProperty2":{}},"numeric_detection":true,"properties":{},"_routing":{"required":true},"_size":{"enabled":true},"_source":{"compress":true,"compress_threshold":"string","enabled":true,"excludes":["string"],"includes":["string"],"mode":"disabled"},"runtime":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"enabled":true,"subobjects":"true","_data_stream_timestamp":{"enabled":true}},"settings_override":{},"remove_index_blocks":true}'

Create an index from a source index Technical preview; Added in 8.18.0

POST /_create_from/{source}/{dest}

Copy the mappings and settings from the source index to a destination index while allowing request settings and mappings to override the source values.

Path parameters

  • source string Required

    The source index or data stream name

  • dest string Required

    The destination index or data stream name

application/json

Body Required

  • mappings_override object
    Hide mappings_override attributes Show mappings_override attributes object
    • all_field object
      Hide all_field attributes Show all_field attributes object
      • analyzer string Required
      • enabled boolean Required
      • omit_norms boolean Required
      • search_analyzer string Required
      • similarity string Required
      • store boolean Required
      • store_term_vector_offsets boolean Required
      • store_term_vector_payloads boolean Required
      • store_term_vector_positions boolean Required
      • store_term_vectors boolean Required
    • date_detection boolean
    • dynamic string

      Values are strict, runtime, true, or false.

    • dynamic_date_formats array[string]
    • dynamic_templates array[object]
    • _field_names object
      Hide _field_names attribute Show _field_names attribute object
      • enabled boolean Required
    • index_field object
      Hide index_field attribute Show index_field attribute object
      • enabled boolean Required
    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties
    • numeric_detection boolean
    • properties object
    • _routing object
      Hide _routing attribute Show _routing attribute object
      • required boolean Required
    • _size object
      Hide _size attribute Show _size attribute object
      • enabled boolean Required
    • _source object
      Hide _source attributes Show _source attributes object
      • compress boolean
      • compress_threshold string
      • enabled boolean
      • excludes array[string]
      • includes array[string]
      • mode string

        Values are disabled, stored, or synthetic.

    • runtime object
      Hide runtime attribute Show runtime attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • format string
        • format string

          A custom format for date type runtime fields.

        • input_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_index string
        • script object
          Hide script attributes Show script attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties
          • lang string

            Any of:

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • type string Required

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • enabled boolean
    • subobjects string

      Values are true or false.

    • _data_stream_timestamp object
      Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
      • enabled boolean Required
  • settings_override object
    Index settings
  • remove_index_blocks boolean

    If index blocks should be removed when creating destination index (optional)

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required
    • index string Required
    • shards_acknowledged boolean Required
POST /_create_from/{source}/{dest}
curl \
 --request POST 'http://api.example.com/_create_from/{source}/{dest}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"mappings_override":{"all_field":{"analyzer":"string","enabled":true,"omit_norms":true,"search_analyzer":"string","similarity":"string","store":true,"store_term_vector_offsets":true,"store_term_vector_payloads":true,"store_term_vector_positions":true,"store_term_vectors":true},"date_detection":true,"dynamic":"strict","dynamic_date_formats":["string"],"dynamic_templates":[{}],"_field_names":{"enabled":true},"index_field":{"enabled":true},"_meta":{"additionalProperty1":{},"additionalProperty2":{}},"numeric_detection":true,"properties":{},"_routing":{"required":true},"_size":{"enabled":true},"_source":{"compress":true,"compress_threshold":"string","enabled":true,"excludes":["string"],"includes":["string"],"mode":"disabled"},"runtime":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"enabled":true,"subobjects":"true","_data_stream_timestamp":{"enabled":true}},"settings_override":{},"remove_index_blocks":true}'





























Prepare a node to be shut down Generally available; Added in 7.13.0

PUT /_nodes/{node_id}/shutdown

NOTE: This feature is designed for indirect use by Elastic Cloud, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.

If you specify a node that is offline, it will be prepared for shut down when it rejoins the cluster.

If the operator privileges feature is enabled, you must be an operator to use this API.

The API migrates ongoing tasks and index shards to other nodes as needed to prepare a node to be restarted or shut down and removed from the cluster. This ensures that Elasticsearch can be stopped safely with minimal disruption to the cluster.

You must specify the type of shutdown: restart, remove, or replace. If a node is already being prepared for shutdown, you can use this API to change the shutdown type.

IMPORTANT: This API does NOT terminate the Elasticsearch process. Monitor the node shutdown status to determine when it is safe to stop Elasticsearch.

Required authorization

  • Cluster privileges: manage

Path parameters

  • node_id string Required

    The node identifier. This parameter is not validated against the cluster's active nodes. This enables you to register a node for shut down while it is offline. No error is thrown if you specify an invalid node ID.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are nanos, micros, ms, s, m, h, or d.

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

    Values are nanos, micros, ms, s, m, h, or d.

application/json

Body Required

  • type string Required

    Values are restart, remove, or replace.

  • reason string Required

    A human-readable reason that the node is being shut down. This field provides information for other cluster operators; it does not affect the shut down process.

  • allocation_delay string

    Only valid if type is restart. Controls how long Elasticsearch will wait for the node to restart and join the cluster before reassigning its shards to other nodes. This works the same as delaying allocation with the index.unassigned.node_left.delayed_timeout setting. If you specify both a restart allocation delay and an index-level allocation delay, the longer of the two is used.

  • target_node_name string

    Only valid if type is replace. Specifies the name of the node that is replacing the node being shut down. Shards from the shut down node are only allowed to be allocated to the target node, and no other data will be allocated to the target node. During relocation of data certain allocation rules are ignored, such as disk watermarks or user attribute filtering rules.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

PUT /_nodes/{node_id}/shutdown
curl \
 --request PUT 'http://api.example.com/_nodes/{node_id}/shutdown' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"type\": \"restart\",\n  \"reason\": \"Demonstrating how the node shutdown API works\",\n  \"allocation_delay\": \"20m\"\n}"'
Request example
Register a node for shutdown with `PUT /_nodes/USpTGYaBSIKbgSUJR2Z9lg/shutdown`. The `restart` type prepares the node to be restarted.
{
  "type": "restart",
  "reason": "Demonstrating how the node shutdown API works",
  "allocation_delay": "20m"
}

























Create or update a query ruleset Generally available; Added in 8.10.0

PUT /_query_rules/{ruleset_id}

There is a limit of 100 rules per ruleset. This limit can be increased by using the xpack.applications.rules.max_rules_per_ruleset cluster setting.

IMPORTANT: Due to limitations within pinned queries, you can only select documents using ids or docs, but cannot use both in single rule. It is advised to use one or the other in query rulesets, to avoid errors. Additionally, pinned queries have a maximum limit of 100 pinned hits. If multiple matching rules pin more than 100 documents, only the first 100 documents are pinned in the order they are specified in the ruleset.

Required authorization

  • Cluster privileges: manage_search_query_rules
External documentation

Path parameters

  • ruleset_id string Required

    The unique identifier of the query ruleset to be created or updated.

application/json

Body Required

  • rules object | array[object]

    One of:
    Hide attributes Show attributes
    • rule_id string Required
    • type string Required

      Values are pinned or exclude.

    • criteria object | array[object] Required

      The criteria that must be met for the rule to be applied. If multiple criteria are specified for a rule, all criteria must be met for the rule to be applied.

      One of:
      Hide attributes Show attributes
      • type string Required

        Values are global, exact, exact_fuzzy, fuzzy, prefix, suffix, contains, lt, lte, gt, gte, or always.

      • metadata string

        The metadata field to match against. This metadata will be used to match against match_criteria sent in the rule. It is required for all criteria types except always.

      • values array[object]

        The values to match against the metadata field. Only one value must match for the criteria to be met. It is required for all criteria types except always.

    • actions object Required
      Hide actions attributes Show actions attributes object
      • ids array[string]

        The unique document IDs of the documents to apply the rule to. Only one of ids or docs may be specified and at least one must be specified.

      • docs array[object]

        The documents to apply the rule to. Only one of ids or docs may be specified and at least one must be specified. There is a maximum value of 100 documents in a rule. You can specify the following attributes for each document:

        • _index: The index of the document to pin.
        • _id: The unique document ID.
        Hide docs attributes Show docs attributes object
        • _id string Required
        • _index string Required
    • priority number

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

PUT /_query_rules/{ruleset_id}
curl \
 --request PUT 'http://api.example.com/_query_rules/{ruleset_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"rules\": [\n        {\n            \"rule_id\": \"my-rule1\",\n            \"type\": \"pinned\",\n            \"criteria\": [\n                {\n                    \"type\": \"contains\",\n                    \"metadata\": \"user_query\",\n                    \"values\": [ \"pugs\", \"puggles\" ]\n                },\n                {\n                    \"type\": \"exact\",\n                    \"metadata\": \"user_country\",\n                    \"values\": [ \"us\" ]\n                }\n            ],\n            \"actions\": {\n                \"ids\": [\n                    \"id1\",\n                    \"id2\"\n                ]\n            }\n        },\n        {\n            \"rule_id\": \"my-rule2\",\n            \"type\": \"pinned\",\n            \"criteria\": [\n                {\n                    \"type\": \"fuzzy\",\n                    \"metadata\": \"user_query\",\n                    \"values\": [ \"rescue dogs\" ]\n                }\n            ],\n            \"actions\": {\n                \"docs\": [\n                    {\n                        \"_index\": \"index1\",\n                        \"_id\": \"id3\"\n                    },\n                    {\n                        \"_index\": \"index2\",\n                        \"_id\": \"id4\"\n                    }\n                ]\n            }\n        }\n    ]\n}"'
Request example
Run `PUT _query_rules/my-ruleset` to create a new query ruleset. Two rules are associated with `my-ruleset`. `my-rule1` will pin documents with IDs `id1` and `id2` when `user_query` contains `pugs` or `puggles` and `user_country` exactly matches `us`. `my-rule2` will exclude documents from different specified indices with IDs `id3` and `id4` when the `query_string` fuzzily matches `rescue dogs`.
{
    "rules": [
        {
            "rule_id": "my-rule1",
            "type": "pinned",
            "criteria": [
                {
                    "type": "contains",
                    "metadata": "user_query",
                    "values": [ "pugs", "puggles" ]
                },
                {
                    "type": "exact",
                    "metadata": "user_country",
                    "values": [ "us" ]
                }
            ],
            "actions": {
                "ids": [
                    "id1",
                    "id2"
                ]
            }
        },
        {
            "rule_id": "my-rule2",
            "type": "pinned",
            "criteria": [
                {
                    "type": "fuzzy",
                    "metadata": "user_query",
                    "values": [ "rescue dogs" ]
                }
            ],
            "actions": {
                "docs": [
                    {
                        "_index": "index1",
                        "_id": "id3"
                    },
                    {
                        "_index": "index2",
                        "_id": "id4"
                    }
                ]
            }
        }
    ]
}




Get all query rulesets Generally available; Added in 8.10.0

GET /_query_rules

Get summarized information about the query rulesets.

Required authorization

  • Cluster privileges: manage_search_query_rules

Query parameters

  • from number

    The offset from the first result to fetch.

  • size number

    The maximum number of results to retrieve.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • results array[object] Required
      Hide results attributes Show results attributes object
      • ruleset_id string Required
      • rule_total_count number Required

        The number of rules associated with the ruleset.

      • rule_criteria_types_counts object Required

        A map of criteria type (for example, exact) to the number of rules of that type.

        NOTE: The counts in rule_criteria_types_counts may be larger than the value of rule_total_count because a rule may have multiple criteria.

        Hide rule_criteria_types_counts attribute Show rule_criteria_types_counts attribute object
        • * number Additional properties
      • rule_type_counts object Required

        A map of rule type (for example, pinned) to the number of rules of that type.

        Hide rule_type_counts attribute Show rule_type_counts attribute object
        • * number Additional properties
GET /_query_rules
curl \
 --request GET 'http://api.example.com/_query_rules' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _query_rules/?from=0&size=3`.
{
    "count": 3,
    "results": [
        {
            "ruleset_id": "ruleset-1",
            "rule_total_count": 1,
            "rule_criteria_types_counts": {
                "exact": 1
            }
        },
        {
            "ruleset_id": "ruleset-2",
            "rule_total_count": 2,
            "rule_criteria_types_counts": {
                "exact": 1,
                "fuzzy": 1
            }
        },
        {
            "ruleset_id": "ruleset-3",
            "rule_total_count": 3,
            "rule_criteria_types_counts": {
                "exact": 1,
                "fuzzy": 2
            }
        }
    ]
}