The API accepts 3 different authentication methods:
Elasticsearch APIs support key-based authentication. You must create an API key and use the encoded value in the request header. For example:
curl -X GET "${ES_URL}/_cat/indices?v=true" \
-H "Authorization: ApiKey ${API_KEY}"
To get API keys, use the /_security/api_key APIs.
Elasticsearch APIs support the use of bearer tokens in the Authorization HTTP header to authenticate with the API.
For examples, refer to Token-based authentication services
NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are -1 or 0.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Values are -1 or 0.
DELETE /_autoscaling/policy/*
resp = client.autoscaling.delete_autoscaling_policy(
name="*",
)const response = await client.autoscaling.deleteAutoscalingPolicy({
name: "*",
});response = client.autoscaling.delete_autoscaling_policy(
name: "*"
)$resp = $client->autoscaling()->deleteAutoscalingPolicy([
"name" => "*",
]);curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_autoscaling/policy/*"client.autoscaling().deleteAutoscalingPolicy(d -> d
.name("*")
);
{
"acknowledged": true
}All methods and paths for this operation:
GET _application/analytics/my*
resp = client.search_application.get_behavioral_analytics(
name="my*",
)const response = await client.searchApplication.getBehavioralAnalytics({
name: "my*",
});response = client.search_application.get_behavioral_analytics(
name: "my*"
)$resp = $client->searchApplication()->getBehavioralAnalytics([
"name" => "my*",
]);curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_application/analytics/my*"client.searchApplication().getBehavioralAnalytics(g -> g
.name("my*")
);
{
"my_analytics_collection": {
"event_data_stream": {
"name": "behavioral_analytics-events-my_analytics_collection"
}
},
"my_analytics_collection2": {
"event_data_stream": {
"name": "behavioral_analytics-events-my_analytics_collection2"
}
}
}The name of the behavioral analytics collection.
The analytics event type.
Values are page_view, search, or search_click.
POST _application/analytics/my_analytics_collection/event/search_click
{
"session": {
"id": "1797ca95-91c9-4e2e-b1bd-9c38e6f386a9"
},
"user": {
"id": "5f26f01a-bbee-4202-9298-81261067abbd"
},
"search":{
"query": "search term",
"results": {
"items": [
{
"document": {
"id": "123",
"index": "products"
}
}
],
"total_results": 10
},
"sort": {
"name": "relevance"
},
"search_application": "website"
},
"document":{
"id": "123",
"index": "products"
}
}resp = client.search_application.post_behavioral_analytics_event(
collection_name="my_analytics_collection",
event_type="search_click",
payload={
"session": {
"id": "1797ca95-91c9-4e2e-b1bd-9c38e6f386a9"
},
"user": {
"id": "5f26f01a-bbee-4202-9298-81261067abbd"
},
"search": {
"query": "search term",
"results": {
"items": [
{
"document": {
"id": "123",
"index": "products"
}
}
],
"total_results": 10
},
"sort": {
"name": "relevance"
},
"search_application": "website"
},
"document": {
"id": "123",
"index": "products"
}
},
)const response = await client.searchApplication.postBehavioralAnalyticsEvent({
collection_name: "my_analytics_collection",
event_type: "search_click",
payload: {
session: {
id: "1797ca95-91c9-4e2e-b1bd-9c38e6f386a9",
},
user: {
id: "5f26f01a-bbee-4202-9298-81261067abbd",
},
search: {
query: "search term",
results: {
items: [
{
document: {
id: "123",
index: "products",
},
},
],
total_results: 10,
},
sort: {
name: "relevance",
},
search_application: "website",
},
document: {
id: "123",
index: "products",
},
},
});response = client.search_application.post_behavioral_analytics_event(
collection_name: "my_analytics_collection",
event_type: "search_click",
body: {
"session": {
"id": "1797ca95-91c9-4e2e-b1bd-9c38e6f386a9"
},
"user": {
"id": "5f26f01a-bbee-4202-9298-81261067abbd"
},
"search": {
"query": "search term",
"results": {
"items": [
{
"document": {
"id": "123",
"index": "products"
}
}
],
"total_results": 10
},
"sort": {
"name": "relevance"
},
"search_application": "website"
},
"document": {
"id": "123",
"index": "products"
}
}
)$resp = $client->searchApplication()->postBehavioralAnalyticsEvent([
"collection_name" => "my_analytics_collection",
"event_type" => "search_click",
"body" => [
"session" => [
"id" => "1797ca95-91c9-4e2e-b1bd-9c38e6f386a9",
],
"user" => [
"id" => "5f26f01a-bbee-4202-9298-81261067abbd",
],
"search" => [
"query" => "search term",
"results" => [
"items" => array(
[
"document" => [
"id" => "123",
"index" => "products",
],
],
),
"total_results" => 10,
],
"sort" => [
"name" => "relevance",
],
"search_application" => "website",
],
"document" => [
"id" => "123",
"index" => "products",
],
],
]);curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"session":{"id":"1797ca95-91c9-4e2e-b1bd-9c38e6f386a9"},"user":{"id":"5f26f01a-bbee-4202-9298-81261067abbd"},"search":{"query":"search term","results":{"items":[{"document":{"id":"123","index":"products"}}],"total_results":10},"sort":{"name":"relevance"},"search_application":"website"},"document":{"id":"123","index":"products"}}' "$ELASTICSEARCH_URL/_application/analytics/my_analytics_collection/event/search_click"client.searchApplication().postBehavioralAnalyticsEvent(p -> p
.collectionName("my_analytics_collection")
.eventType(EventType.SearchClick)
.payload(JsonData.fromJson("{\"session\":{\"id\":\"1797ca95-91c9-4e2e-b1bd-9c38e6f386a9\"},\"user\":{\"id\":\"5f26f01a-bbee-4202-9298-81261067abbd\"},\"search\":{\"query\":\"search term\",\"results\":{\"items\":[{\"document\":{\"id\":\"123\",\"index\":\"products\"}}],\"total_results\":10},\"sort\":{\"name\":\"relevance\"},\"search_application\":\"website\"},\"document\":{\"id\":\"123\",\"index\":\"products\"}}"))
);
{
"session": {
"id": "1797ca95-91c9-4e2e-b1bd-9c38e6f386a9"
},
"user": {
"id": "5f26f01a-bbee-4202-9298-81261067abbd"
},
"search":{
"query": "search term",
"results": {
"items": [
{
"document": {
"id": "123",
"index": "products"
}
}
],
"total_results": 10
},
"sort": {
"name": "relevance"
},
"search_application": "website"
},
"document":{
"id": "123",
"index": "products"
}
}The compact and aligned text (CAT) APIs aim are intended only for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, it's recommend to use a corresponding JSON API.
All the cat commands accept a query string parameter help to see all the headers and info they provide, and the /_cat command alone lists all the available commands.
All methods and paths for this operation:
Get a snapshot of the number of shards allocated to each data node and their disk space.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications.
monitorA comma-separated list of node identifiers or names used to limit the returned information.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
Period to wait for a connection to the master node.
Values are -1 or 0.
GET /_cat/allocation?v=true&format=json
resp = client.cat.allocation(
v=True,
format="json",
)const response = await client.cat.allocation({
v: "true",
format: "json",
});response = client.cat.allocation(
v: "true",
format: "json"
)$resp = $client->cat()->allocation([
"v" => "true",
"format" => "json",
]);curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cat/allocation?v=true&format=json"client.cat().allocation();
[
{
"shards": "1",
"shards.undesired": "0",
"write_load.forecast": "0.0",
"disk.indices.forecast": "260b",
"disk.indices": "260b",
"disk.used": "47.3gb",
"disk.avail": "43.4gb",
"disk.total": "100.7gb",
"disk.percent": "46",
"host": "127.0.0.1",
"ip": "127.0.0.1",
"node": "CSUXak2",
"node.role": "himrst"
}
]IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console.
They are not intended for use by applications. For application consumption, use the cluster health API.
This API is often used to check malfunctioning clusters.
To help you track cluster health alongside log files and alerting systems, the API returns timestamps in two formats:
HH:MM:SS, which is human-readable but includes no date information;
Unix epoch time, which is machine-sortable and includes date information.
The latter format is useful for cluster recoveries that take multiple days.
You can use the cat health API to verify cluster health across multiple nodes.
You also can use the API to track the recovery of a large cluster over a longer period of time.
monitorThe unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
If true, returns HH:MM:SS and Unix epoch timestamps.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
GET /_cat/health?v=true&format=json
resp = client.cat.health(
v=True,
format="json",
)const response = await client.cat.health({
v: "true",
format: "json",
});response = client.cat.health(
v: "true",
format: "json"
)$resp = $client->cat()->health([
"v" => "true",
"format" => "json",
]);curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cat/health?v=true&format=json"client.cat().health();
[
{
"epoch": "1475871424",
"timestamp": "16:17:04",
"cluster": "elasticsearch",
"status": "green",
"node.total": "1",
"node.data": "1",
"shards": "1",
"pri": "1",
"relo": "0",
"init": "0",
"unassign": "0",
"unassign.pri": "0",
"pending_tasks": "0",
"max_task_wait_time": "-",
"active_shards_percent": "100.0%"
}
]curl \
--request GET 'http://api.example.com/_cat' \
--header "Authorization: $API_KEY"Get information about the master node, including the ID, bound IP address, and name.
IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes info API.
monitorList of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
Period to wait for a connection to the master node.
Values are -1 or 0.
GET /_cat/master?v=true&format=json
resp = client.cat.master(
v=True,
format="json",
)const response = await client.cat.master({
v: "true",
format: "json",
});response = client.cat.master(
v: "true",
format: "json"
)$resp = $client->cat()->master([
"v" => "true",
"format" => "json",
]);curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cat/master?v=true&format=json"client.cat().master();
[
{
"id": "YzWoH_2BT-6UjVGDyPdqYg",
"host": "127.0.0.1",
"ip": "127.0.0.1",
"node": "YzWoH_2"
}
]All methods and paths for this operation:
Get configuration and usage information about datafeeds.
This API returns a maximum of 10,000 datafeeds.
If the Elasticsearch security features are enabled, you must have monitor_ml, monitor, manage_ml, or manage
cluster privileges to use this API.
IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get datafeed statistics API.
monitor_mlSpecifies what to do when the request:
_all string or no identifiers and there are no matches.If true, the API returns an empty datafeeds array when there are no matches and the subset of results when
there are partial matches. If false, the API returns a 404 status code when there are no matches or only
partial matches.
Comma-separated list of column names to display.
Supported values include:
ae (or assignment_explanation): For started datafeeds only, contains messages relating to the selection of
a node.bc (or buckets.count, bucketsCount): The number of buckets processed.id: A numerical character string that uniquely identifies the datafeed.na (or node.address, nodeAddress): For started datafeeds only, the network address of the node where the
datafeed is started.ne (or node.ephemeral_id, nodeEphemeralId): For started datafeeds only, the ephemeral ID of the node where the
datafeed is started.ni (or node.id, nodeId): For started datafeeds only, the unique identifier of the node where the
datafeed is started.nn (or node.name, nodeName): For started datafeeds only, the name of the node where the datafeed is
started.sba (or search.bucket_avg, searchBucketAvg): The average search time per bucket, in milliseconds.sc (or search.count, searchCount): The number of searches run by the datafeed.seah (or search.exp_avg_hour, searchExpAvgHour): The exponential average search time per hour, in milliseconds.st (or search.time, searchTime): The total time the datafeed spent searching, in milliseconds.s (or state): The status of the datafeed: starting, started, stopping, or stopped.
If starting, the datafeed has been requested to start but has not yet
started. If started, the datafeed is actively receiving data. If
stopping, the datafeed has been requested to stop gracefully and is
completing its final action. If stopped, the datafeed is stopped and will
not receive data until it is re-started.Comma-separated list of column names or column aliases used to sort the response.
Supported values include:
ae (or assignment_explanation): For started datafeeds only, contains messages relating to the selection of
a node.bc (or buckets.count, bucketsCount): The number of buckets processed.id: A numerical character string that uniquely identifies the datafeed.na (or node.address, nodeAddress): For started datafeeds only, the network address of the node where the
datafeed is started.ne (or node.ephemeral_id, nodeEphemeralId): For started datafeeds only, the ephemeral ID of the node where the
datafeed is started.ni (or node.id, nodeId): For started datafeeds only, the unique identifier of the node where the
datafeed is started.nn (or node.name, nodeName): For started datafeeds only, the name of the node where the datafeed is
started.sba (or search.bucket_avg, searchBucketAvg): The average search time per bucket, in milliseconds.sc (or search.count, searchCount): The number of searches run by the datafeed.seah (or search.exp_avg_hour, searchExpAvgHour): The exponential average search time per hour, in milliseconds.st (or search.time, searchTime): The total time the datafeed spent searching, in milliseconds.s (or state): The status of the datafeed: starting, started, stopping, or stopped.
If starting, the datafeed has been requested to start but has not yet
started. If started, the datafeed is actively receiving data. If
stopping, the datafeed has been requested to stop gracefully and is
completing its final action. If stopped, the datafeed is stopped and will
not receive data until it is re-started.The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
GET _cat/ml/datafeeds?v=true&format=json
resp = client.cat.ml_datafeeds(
v=True,
format="json",
)const response = await client.cat.mlDatafeeds({
v: "true",
format: "json",
});response = client.cat.ml_datafeeds(
v: "true",
format: "json"
)$resp = $client->cat()->mlDatafeeds([
"v" => "true",
"format" => "json",
]);curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cat/ml/datafeeds?v=true&format=json"client.cat().mlDatafeeds();
[
{
"id": "datafeed-high_sum_total_sales",
"state": "stopped",
"buckets.count": "743",
"search.count": "7"
},
{
"id": "datafeed-low_request_rate",
"state": "stopped",
"buckets.count": "1457",
"search.count": "3"
},
{
"id": "datafeed-response_code_rates",
"state": "stopped",
"buckets.count": "1460",
"search.count": "18"
},
{
"id": "datafeed-url_scanning",
"state": "stopped",
"buckets.count": "1460",
"search.count": "18"
}
]List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
Period to wait for a connection to the master node.
Values are -1 or 0.
GET /_cat/nodeattrs?v=true&format=json
resp = client.cat.nodeattrs(
v=True,
format="json",
)const response = await client.cat.nodeattrs({
v: "true",
format: "json",
});response = client.cat.nodeattrs(
v: "true",
format: "json"
)$resp = $client->cat()->nodeattrs([
"v" => "true",
"format" => "json",
]);curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cat/nodeattrs?v=true&format=json"client.cat().nodeattrs();
[
{
"node": "node-0",
"host": "127.0.0.1",
"ip": "127.0.0.1",
"attr": "testattr",
"value": "test"
}
][
{
"name": "node-0",
"pid": "19566",
"attr": "testattr",
"value": "test"
}
]The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
If true, return the full node ID. If false, return the shortened node ID.
If true, the response includes information from segments that are not loaded into memory.
A comma-separated list of columns names to display. It supports simple wildcards.
Supported values include:
build (or b): The Elasticsearch build hash. For example: 5c03844.completion.size (or cs, completionSize): The size of completion. For example: 0b.cpu: The percentage of recent system CPU used.disk.avail (or d, disk, diskAvail): The available disk space. For example: 198.4gb.disk.total (or dt, diskTotal): The total disk space. For example: 458.3gb.disk.used (or du, diskUsed): The used disk space. For example: 259.8gb.disk.used_percent (or dup, diskUsedPercent): The percentage of disk space used.fielddata.evictions (or fe, fielddataEvictions): The number of fielddata cache evictions.fielddata.memory_size (or fm, fielddataMemory): The fielddata cache memory used. For example: 0b.file_desc.current (or fdc, fileDescriptorCurrent): The number of file descriptors used.file_desc.max (or fdm, fileDescriptorMax): The maximum number of file descriptors.file_desc.percent (or fdp, fileDescriptorPercent): The percentage of file descriptors used.flush.total (or ft, flushTotal): The number of flushes.flush.total_time (or ftt, flushTotalTime): The amount of time spent in flush.get.current (or gc, getCurrent): The number of current get operations.get.exists_time (or geti, getExistsTime): The time spent in successful get operations. For example: 14ms.get.exists_total (or geto, getExistsTotal): The number of successful get operations.get.missing_time (or gmti, getMissingTime): The time spent in failed get operations. For example: 0s.get.missing_total (or gmto, getMissingTotal): The number of failed get operations.get.time (or gti, getTime): The amount of time spent in get operations. For example: 14ms.get.total (or gto, getTotal): The number of get operations.heap.current (or hc, heapCurrent): The used heap size. For example: 311.2mb.heap.max (or hm, heapMax): The total heap size. For example: 4gb.heap.percent (or hp, heapPercent): The used percentage of total allocated Elasticsearch JVM heap.
This value reflects only the Elasticsearch process running within the operating system and is the most direct indicator of its JVM, heap, or memory resource performance.http_address (or http): The bound HTTP address.id (or nodeId): The identifier for the node.indexing.delete_current (or idc, indexingDeleteCurrent): The number of current deletion operations.indexing.delete_time (or idti, indexingDeleteTime): The time spent in deletion operations. For example: 2ms.indexing.delete_total (or idto, indexingDeleteTotal): The number of deletion operations.indexing.index_current (or iic, indexingIndexCurrent): The number of current indexing operations.indexing.index_failed (or iif, indexingIndexFailed): The number of failed indexing operations.indexing.index_failed_due_to_version_conflict (or iifvc, indexingIndexFailedDueToVersionConflict): The number of indexing operations that failed due to version conflict.indexing.index_time (or iiti, indexingIndexTime): The time spent in indexing operations. For example: 134ms.indexing.index_total (or iito, indexingIndexTotal): The number of indexing operations.ip (or i): The IP address.jdk (or j): The Java version. For example: 1.8.0.load_1m (or l): The most recent load average. For example: 0.22.load_5m (or l): The load average for the last five minutes. For example: 0.78.load_15m (or l): The load average for the last fifteen minutes. For example: 1.24.mappings.total_count (or mtc, mappingsTotalCount): The number of mappings, including runtime and object fields.mappings.total_estimated_overhead_in_bytes (or mteo, mappingsTotalEstimatedOverheadInBytes): The estimated heap overhead, in bytes, of mappings on this node, which allows for 1KiB of heap for every mapped field.master (or m): Indicates whether the node is the elected master node.
Returned values include * (elected master) and - (not elected master).merges.current (or mc, mergesCurrent): The number of current merge operations.merges.current_docs (or mcd, mergesCurrentDocs): The number of current merging documents.merges.current_size (or mcs, mergesCurrentSize): The size of current merges. For example: 0b.merges.total (or mt, mergesTotal): The number of completed merge operations.merges.total_docs (or mtd, mergesTotalDocs): The number of merged documents.merges.total_size (or mts, mergesTotalSize): The total size of merges. For example: 0b.merges.total_time (or mtt, mergesTotalTime): The time spent merging documents. For example: 0s.name (or n): The node name.node.role (or r, role, nodeRole): The roles of the node.
Returned values include c (cold node), d (data node), f (frozen node), h (hot node), i (ingest node), l (machine learning node), m (master-eligible node), r (remote cluster client node), s (content node), t (transform node), v (voting-only node), w (warm node), and - (coordinating node only).
For example, dim indicates a master-eligible data and ingest node.pid (or p): The process identifier.port (or po): The bound transport port number.query_cache.memory_size (or qcm, queryCacheMemory): The used query cache memory. For example: 0b.query_cache.evictions (or qce, queryCacheEvictions): The number of query cache evictions.query_cache.hit_count (or qchc, queryCacheHitCount): The query cache hit count.query_cache.miss_count (or qcmc, queryCacheMissCount): The query cache miss count.ram.current (or rc, ramCurrent): The used total memory. For example: 513.4mb.ram.max (or rm, ramMax): The total memory. For example: 2.9gb.ram.percent (or rp, ramPercent): The used percentage of the total operating system memory.
This reflects all processes running on the operating system instead of only Elasticsearch and is not guaranteed to correlate to its performance.refresh.total (or rto, refreshTotal): The number of refresh operations.refresh.time (or rti, refreshTime): The time spent in refresh operations. For example: 91ms.request_cache.memory_size (or rcm, requestCacheMemory): The used request cache memory. For example: 0b.request_cache.evictions (or rce, requestCacheEvictions): The number of request cache evictions.request_cache.hit_count (or rchc, requestCacheHitCount): The request cache hit count.request_cache.miss_count (or rcmc, requestCacheMissCount): The request cache miss count.script.compilations (or scrcc, scriptCompilations): The number of total script compilations.script.cache_evictions (or scrce, scriptCacheEvictions): The number of total compiled scripts evicted from cache.search.fetch_current (or sfc, searchFetchCurrent): The number of current fetch phase operations.search.fetch_time (or sfti, searchFetchTime): The time spent in fetch phase. For example: 37ms.search.fetch_total (or sfto, searchFetchTotal): The number of fetch operations.search.open_contexts (or so, searchOpenContexts): The number of open search contexts.search.query_current (or sqc, searchQueryCurrent): The number of current query phase operations.search.query_time (or sqti, searchQueryTime): The time spent in query phase. For example: 43ms.search.query_total (or sqto, searchQueryTotal): The number of query operations.search.scroll_current (or scc, searchScrollCurrent): The number of open scroll contexts.search.scroll_time (or scti, searchScrollTime): The amount of time scroll contexts were held open. For example: 2m.search.scroll_total (or scto, searchScrollTotal): The number of completed scroll contexts.segments.count (or sc, segmentsCount): The number of segments.segments.fixed_bitset_memory (or sfbm, fixedBitsetMemory): The memory used by fixed bit sets for nested object field types and type filters for types referred in join fields.
For example: 1.0kb.segments.index_writer_memory (or siwm, segmentsIndexWriterMemory): The memory used by the index writer. For example: 18mb.segments.memory (or sm, segmentsMemory): The memory used by segments. For example: 1.4kb.segments.version_map_memory (or svmm, segmentsVersionMapMemory): The memory used by the version map. For example: 1.0kb.shard_stats.total_count (or sstc, shards, shardStatsTotalCount): The number of shards assigned.suggest.current (or suc, suggestCurrent): The number of current suggest operations.suggest.time (or suti, suggestTime): The time spent in suggest operations.suggest.total (or suto, suggestTotal): The number of suggest operations.uptime (or u): The amount of node uptime. For example: 17.3m.version (or v): The Elasticsearch version. For example: 9.0.0.A comma-separated list of column names or aliases that determines the sort order.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
The period to wait for a connection to the master node.
Values are -1 or 0.
The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
GET /_cat/nodes?v=true&h=id,ip,port,v,m&format=json
resp = client.cat.nodes(
v=True,
h="id,ip,port,v,m",
format="json",
)const response = await client.cat.nodes({
v: "true",
h: "id,ip,port,v,m",
format: "json",
});response = client.cat.nodes(
v: "true",
h: "id,ip,port,v,m",
format: "json"
)$resp = $client->cat()->nodes([
"v" => "true",
"h" => "id,ip,port,v,m",
"format" => "json",
]);curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cat/nodes?v=true&h=id,ip,port,v,m&format=json"client.cat().nodes();
[
{
"ip": "127.0.0.1",
"heap.percent": "65",
"ram.percent": "99",
"cpu": "42",
"load_1m": "3.07",
"load_5m": null,
"load_15m": null,
"node.role": "cdfhilmrstw",
"master": "*",
"name": "mJw06l1"
}
][
{
"id": "veJR",
"ip": "127.0.0.1",
"port": "59938",
"v": "9.0.0",
"m": "*"
}
]All methods and paths for this operation:
Get configuration and usage information about transforms.
CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get transform statistics API.
monitor_transformA transform identifier or a wildcard expression. If you do not specify one of these options, the API returns information for all transforms.
Specifies what to do when the request: contains wildcard expressions and there are no transforms that match; contains the _all string or no identifiers and there are no matches; contains wildcard expressions and there are only partial matches.
If true, it returns an empty transforms array when there are no matches and the subset of results when there are partial matches.
If false, the request returns a 404 status code when there are no matches or only partial matches.
Skips the specified number of transforms.
Comma-separated list of column names to display.
Supported values include:
changes_last_detection_time (or cldt): The timestamp when changes were last detected in the source indices.checkpoint (or cp): The sequence number for the checkpoint.checkpoint_duration_time_exp_avg (or cdtea, checkpointTimeExpAvg): Exponential moving average of the duration of the checkpoint, in
milliseconds.checkpoint_progress (or c, checkpointProgress): The progress of the next checkpoint that is currently in progress.create_time (or ct, createTime): The time the transform was created.delete_time (or dtime): The amount of time spent deleting, in milliseconds.description (or d): The description of the transform.dest_index (or di, destIndex): The destination index for the transform. The mappings of the destination
index are deduced based on the source fields when possible. If alternate
mappings are required, use the Create index API prior to starting the
transform.documents_deleted (or docd): The number of documents that have been deleted from the destination index
due to the retention policy for this transform.documents_indexed (or doci): The number of documents that have been indexed into the destination index
for the transform.docs_per_second (or dps): Specifies a limit on the number of input documents per second. This setting
throttles the transform by adding a wait time between search requests. The
default value is null, which disables throttling.documents_processed (or docp): The number of documents that have been processed from the source index of
the transform.frequency (or f): The interval between checks for changes in the source indices when the
transform is running continuously. Also determines the retry interval in
the event of transient failures while the transform is searching or
indexing. The minimum value is 1s and the maximum is 1h. The default
value is 1m.id: Identifier for the transform.index_failure (or if): The number of indexing failures.index_time (or itime): The amount of time spent indexing, in milliseconds.index_total (or it): The number of index operations.indexed_documents_exp_avg (or idea): Exponential moving average of the number of new documents that have been
indexed.last_search_time (or lst, lastSearchTime): The timestamp of the last search in the source indices. This field is only
shown if the transform is running.max_page_search_size (or mpsz): Defines the initial page size to use for the composite aggregation for each
checkpoint. If circuit breaker exceptions occur, the page size is
dynamically adjusted to a lower value. The minimum value is 10 and the
maximum is 65,536. The default value is 500.pages_processed (or pp): The number of search or bulk index operations processed. Documents are
processed in batches instead of individually.pipeline (or p): The unique identifier for an ingest pipeline.processed_documents_exp_avg (or pdea): Exponential moving average of the number of documents that have been
processed.processing_time (or pt): The amount of time spent processing results, in milliseconds.reason (or r): If a transform has a failed state, this property provides details about
the reason for the failure.search_failure (or sf): The number of search failures.search_time (or stime): The amount of time spent searching, in milliseconds.search_total (or st): The number of search operations on the source index for the transform.source_index (or si, sourceIndex): The source indices for the transform. It can be a single index, an index
pattern (for example, "my-index-*"), an array of indices (for example,
["my-index-000001", "my-index-000002"]), or an array of index patterns
(for example, ["my-index-*", "my-other-index-*"]. For remote indices use
the syntax "remote_name:index_name". If any indices are in remote
clusters then the master node and at least one transform node must have the
remote_cluster_client node role.state (or s): The status of the transform, which can be one of the following values:
aborting: The transform is aborting.failed: The transform failed. For more information about the failure,
check the reason field.indexing: The transform is actively processing data and creating new
documents.started: The transform is running but not actively indexing data.stopped: The transform is stopped.stopping: The transform is stopping.transform_type (or tt): Indicates the type of transform: batch or continuous.
trigger_count (or tc): The number of times the transform has been triggered by the scheduler. For
example, the scheduler triggers the transform indexer to check for updates
or ingest new data at an interval specified in the frequency property.
version (or v): The version of Elasticsearch that existed on the node when the transform
was created.
Comma-separated list of column names or column aliases used to sort the response.
Supported values include:
changes_last_detection_time (or cldt): The timestamp when changes were last detected in the source indices.checkpoint (or cp): The sequence number for the checkpoint.checkpoint_duration_time_exp_avg (or cdtea, checkpointTimeExpAvg): Exponential moving average of the duration of the checkpoint, in
milliseconds.checkpoint_progress (or c, checkpointProgress): The progress of the next checkpoint that is currently in progress.create_time (or ct, createTime): The time the transform was created.delete_time (or dtime): The amount of time spent deleting, in milliseconds.description (or d): The description of the transform.dest_index (or di, destIndex): The destination index for the transform. The mappings of the destination
index are deduced based on the source fields when possible. If alternate
mappings are required, use the Create index API prior to starting the
transform.documents_deleted (or docd): The number of documents that have been deleted from the destination index
due to the retention policy for this transform.documents_indexed (or doci): The number of documents that have been indexed into the destination index
for the transform.docs_per_second (or dps): Specifies a limit on the number of input documents per second. This setting
throttles the transform by adding a wait time between search requests. The
default value is null, which disables throttling.documents_processed (or docp): The number of documents that have been processed from the source index of
the transform.frequency (or f): The interval between checks for changes in the source indices when the
transform is running continuously. Also determines the retry interval in
the event of transient failures while the transform is searching or
indexing. The minimum value is 1s and the maximum is 1h. The default
value is 1m.id: Identifier for the transform.index_failure (or if): The number of indexing failures.index_time (or itime): The amount of time spent indexing, in milliseconds.index_total (or it): The number of index operations.indexed_documents_exp_avg (or idea): Exponential moving average of the number of new documents that have been
indexed.last_search_time (or lst, lastSearchTime): The timestamp of the last search in the source indices. This field is only
shown if the transform is running.max_page_search_size (or mpsz): Defines the initial page size to use for the composite aggregation for each
checkpoint. If circuit breaker exceptions occur, the page size is
dynamically adjusted to a lower value. The minimum value is 10 and the
maximum is 65,536. The default value is 500.pages_processed (or pp): The number of search or bulk index operations processed. Documents are
processed in batches instead of individually.pipeline (or p): The unique identifier for an ingest pipeline.processed_documents_exp_avg (or pdea): Exponential moving average of the number of documents that have been
processed.processing_time (or pt): The amount of time spent processing results, in milliseconds.reason (or r): If a transform has a failed state, this property provides details about
the reason for the failure.search_failure (or sf): The number of search failures.search_time (or stime): The amount of time spent searching, in milliseconds.search_total (or st): The number of search operations on the source index for the transform.source_index (or si, sourceIndex): The source indices for the transform. It can be a single index, an index
pattern (for example, "my-index-*"), an array of indices (for example,
["my-index-000001", "my-index-000002"]), or an array of index patterns
(for example, ["my-index-*", "my-other-index-*"]. For remote indices use
the syntax "remote_name:index_name". If any indices are in remote
clusters then the master node and at least one transform node must have the
remote_cluster_client node role.state (or s): The status of the transform, which can be one of the following values:
aborting: The transform is aborting.failed: The transform failed. For more information about the failure,
check the reason field.indexing: The transform is actively processing data and creating new
documents.started: The transform is running but not actively indexing data.stopped: The transform is stopped.stopping: The transform is stopping.transform_type (or tt): Indicates the type of transform: batch or continuous.
trigger_count (or tc): The number of times the transform has been triggered by the scheduler. For
example, the scheduler triggers the transform indexer to check for updates
or ingest new data at an interval specified in the frequency property.
version (or v): The version of Elasticsearch that existed on the node when the transform
was created.
The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
The maximum number of transforms to obtain.
GET /_cat/transforms?v=true&format=json
resp = client.cat.transforms(
v=True,
format="json",
)const response = await client.cat.transforms({
v: "true",
format: "json",
});response = client.cat.transforms(
v: "true",
format: "json"
)$resp = $client->cat()->transforms([
"v" => "true",
"format" => "json",
]);curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cat/transforms?v=true&format=json"client.cat().transforms();
[
{
"id" : "ecommerce_transform",
"state" : "started",
"checkpoint" : "1",
"documents_processed" : "705",
"checkpoint_progress" : "100.00",
"changes_last_detection_time" : null
}
]Update the cluster voting config exclusions by node IDs or node names. By default, if there are more than three master-eligible nodes in the cluster and you remove fewer than half of the master-eligible nodes in the cluster at once, the voting configuration automatically shrinks. If you want to shrink the voting configuration to contain fewer than three nodes or to remove half or more of the master-eligible nodes in the cluster at once, use this API to remove departing nodes from the voting configuration manually. The API adds an entry for each specified node to the cluster’s voting configuration exclusions list. It then waits until the cluster has reconfigured its voting configuration to exclude the specified nodes.
Clusters should have no voting configuration exclusions in normal operation.
Once the excluded nodes have stopped, clear the voting configuration exclusions with DELETE /_cluster/voting_config_exclusions.
This API waits for the nodes to be fully removed from the cluster before it returns.
If your cluster has voting configuration exclusions for nodes that you no longer intend to remove, use DELETE /_cluster/voting_config_exclusions?wait_for_removal=false to clear the voting configuration exclusions without waiting for the nodes to leave the cluster.
A response to POST /_cluster/voting_config_exclusions with an HTTP status code of 200 OK guarantees that the node has been removed from the voting configuration and will not be reinstated until the voting configuration exclusions are cleared by calling DELETE /_cluster/voting_config_exclusions.
If the call to POST /_cluster/voting_config_exclusions fails or returns a response with an HTTP status code other than 200 OK then the node may not have been removed from the voting configuration.
In that case, you may safely retry the call.
NOTE: Voting exclusions are required only when you remove at least half of the master-eligible nodes from a cluster in a short time period. They are not required when removing master-ineligible nodes or when removing fewer than half of the master-eligible nodes.
A comma-separated list of the names of the nodes to exclude from the voting configuration. If specified, you may not also specify node_ids.
A comma-separated list of the persistent ids of the nodes to exclude from the voting configuration. If specified, you may not also specify node_names.
Period to wait for a connection to the master node.
Values are -1 or 0.
When adding a voting configuration exclusion, the API waits for the specified nodes to be excluded from the voting configuration before returning. If the timeout expires before the appropriate condition is satisfied, the request fails and returns an error.
Values are -1 or 0.
curl \
--request POST 'http://api.example.com/_cluster/voting_config_exclusions' \
--header "Authorization: $API_KEY"Returns basic information about the cluster.
GET /_info/_all
resp = client.cluster.info(
target="_all",
)const response = await client.cluster.info({
target: "_all",
});response = client.cluster.info(
target: "_all"
)$resp = $client->cluster()->info([
"target" => "_all",
]);curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_info/_all"client.cluster().info(i -> i
.target("_all")
);
Manually change the allocation of individual shards in the cluster. For example, a shard can be moved from one node to another explicitly, an allocation can be canceled, and an unassigned shard can be explicitly allocated to a specific node.
It is important to note that after processing any reroute commands Elasticsearch will perform rebalancing as normal (respecting the values of settings such as cluster.routing.rebalance.enable) in order to remain in a balanced state.
For example, if the requested allocation includes moving a shard from node1 to node2 then this may cause a shard to be moved from node2 back to node1 to even things out.
The cluster can be set to disable allocations using the cluster.routing.allocation.enable setting.
If allocations are disabled then the only allocations that will be performed are explicit ones given using the reroute command, and consequent allocations due to rebalancing.
The cluster will attempt to allocate a shard a maximum of index.allocation.max_retries times in a row (defaults to 5), before giving up and leaving the shard unallocated.
This scenario can be caused by structural problems such as having an analyzer which refers to a stopwords file which doesn’t exist on all nodes.
Once the problem has been corrected, allocation can be manually retried by calling the reroute API with the ?retry_failed URI query parameter, which will attempt a single retry round for these shards.
If true, then the request simulates the operation. It will calculate the result of applying the commands to the current cluster state and return the resulting cluster state after the commands (and rebalancing) have been applied; it will not actually perform the requested changes.
If true, then the response contains an explanation of why the commands can or cannot run.
Limits the information returned to the specified metrics.
If true, then retries allocation of shards that are blocked due to too many subsequent allocation failures.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are -1 or 0.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Values are -1 or 0.
POST /_cluster/reroute?metric=none
{
"commands": [
{
"move": {
"index": "test", "shard": 0,
"from_node": "node1", "to_node": "node2"
}
},
{
"allocate_replica": {
"index": "test", "shard": 1,
"node": "node3"
}
}
]
}resp = client.cluster.reroute(
metric="none",
commands=[
{
"move": {
"index": "test",
"shard": 0,
"from_node": "node1",
"to_node": "node2"
}
},
{
"allocate_replica": {
"index": "test",
"shard": 1,
"node": "node3"
}
}
],
)const response = await client.cluster.reroute({
metric: "none",
commands: [
{
move: {
index: "test",
shard: 0,
from_node: "node1",
to_node: "node2",
},
},
{
allocate_replica: {
index: "test",
shard: 1,
node: "node3",
},
},
],
});response = client.cluster.reroute(
metric: "none",
body: {
"commands": [
{
"move": {
"index": "test",
"shard": 0,
"from_node": "node1",
"to_node": "node2"
}
},
{
"allocate_replica": {
"index": "test",
"shard": 1,
"node": "node3"
}
}
]
}
)$resp = $client->cluster()->reroute([
"metric" => "none",
"body" => [
"commands" => array(
[
"move" => [
"index" => "test",
"shard" => 0,
"from_node" => "node1",
"to_node" => "node2",
],
],
[
"allocate_replica" => [
"index" => "test",
"shard" => 1,
"node" => "node3",
],
],
),
],
]);curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"commands":[{"move":{"index":"test","shard":0,"from_node":"node1","to_node":"node2"}},{"allocate_replica":{"index":"test","shard":1,"node":"node3"}}]}' "$ELASTICSEARCH_URL/_cluster/reroute?metric=none"{
"commands": [
{
"move": {
"index": "test", "shard": 0,
"from_node": "node1", "to_node": "node2"
}
},
{
"allocate_replica": {
"index": "test", "shard": 1,
"node": "node3"
}
}
]
}All methods and paths for this operation:
Get comprehensive information about the state of the cluster.
The cluster state is an internal data structure which keeps track of a variety of information needed by every node, including the identity and attributes of the other nodes in the cluster; cluster-wide settings; index metadata, including the mapping and settings for each index; the location and status of every shard copy in the cluster.
The elected master node ensures that every node in the cluster has a copy of the same cluster state. This API lets you retrieve a representation of this internal state for debugging or diagnostic purposes. You may need to consult the Elasticsearch source code to determine the precise meaning of the response.
By default the API will route requests to the elected master node since this node is the authoritative source of cluster states.
You can also retrieve the cluster state held on the node handling the API request by adding the ?local=true query parameter.
Elasticsearch may need to expend significant effort to compute a response to this API in larger clusters, and the response may comprise a very large quantity of data. If you use this API repeatedly, your cluster may become unstable.
WARNING: The response is a representation of an internal data structure. Its format is not subject to the same compatibility guarantees as other more stable APIs and may change from version to version. Do not query this API using external monitoring tools. Instead, obtain the information you require using other more stable cluster APIs.
monitor,manageLimit the information returned to the specified metrics
A comma-separated list of index names; use _all or empty string to perform the operation on all indices
Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)
Whether to expand wildcard expression to concrete indices that are open, closed or both.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Values are all, open, closed, hidden, or none.
Return settings in flat format (default: false)
Return local information, do not retrieve the state from master node (default: false)
Timeout for waiting for new cluster state in case it is blocked
Values are -1 or 0.
Wait for the metadata version to be equal or greater than the specified metadata version
The maximum time to wait for wait_for_metadata_version before timing out
Values are -1 or 0.
GET /_cluster/state?filter_path=metadata.cluster_coordination.last_committed_config
resp = client.cluster.state(
filter_path="metadata.cluster_coordination.last_committed_config",
)const response = await client.cluster.state({
filter_path: "metadata.cluster_coordination.last_committed_config",
});response = client.cluster.state(
filter_path: "metadata.cluster_coordination.last_committed_config"
)$resp = $client->cluster()->state([
"filter_path" => "metadata.cluster_coordination.last_committed_config",
]);curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_cluster/state?filter_path=metadata.cluster_coordination.last_committed_config"Cancel a connector sync job, which sets the status to cancelling and updates cancellation_requested_at to the current time.
The connector service is then responsible for setting the status of connector sync jobs to cancelled.
PUT _connector/_sync_job/my-connector-sync-job-id/_cancel
resp = client.connector.sync_job_cancel(
connector_sync_job_id="my-connector-sync-job-id",
)const response = await client.connector.syncJobCancel({
connector_sync_job_id: "my-connector-sync-job-id",
});response = client.connector.sync_job_cancel(
connector_sync_job_id: "my-connector-sync-job-id"
)$resp = $client->connector()->syncJobCancel([
"connector_sync_job_id" => "my-connector-sync-job-id",
]);curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_connector/_sync_job/my-connector-sync-job-id/_cancel"client.connector().syncJobCancel(s -> s
.connectorSyncJobId("my-connector-sync-job-id")
);
Set the error field for a connector sync job and set its status to error.
To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for Elastic managed connectors.
PUT _connector/_sync_job/my-connector-sync-job/_error
{
"error": "some-error"
}resp = client.connector.sync_job_error(
connector_sync_job_id="my-connector-sync-job",
error="some-error",
)const response = await client.connector.syncJobError({
connector_sync_job_id: "my-connector-sync-job",
error: "some-error",
});response = client.connector.sync_job_error(
connector_sync_job_id: "my-connector-sync-job",
body: {
"error": "some-error"
}
)$resp = $client->connector()->syncJobError([
"connector_sync_job_id" => "my-connector-sync-job",
"body" => [
"error" => "some-error",
],
]);curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"error":"some-error"}' "$ELASTICSEARCH_URL/_connector/_sync_job/my-connector-sync-job/_error"client.connector().syncJobError(s -> s
.connectorSyncJobId("my-connector-sync-job")
.error("some-error")
);
{
"error": "some-error"
}Update the configuration field in the connector document.
PUT _connector/my-spo-connector/_configuration
{
"values": {
"tenant_id": "my-tenant-id",
"tenant_name": "my-sharepoint-site",
"client_id": "foo",
"secret_value": "bar",
"site_collections": "*"
}
}resp = client.connector.update_configuration(
connector_id="my-spo-connector",
values={
"tenant_id": "my-tenant-id",
"tenant_name": "my-sharepoint-site",
"client_id": "foo",
"secret_value": "bar",
"site_collections": "*"
},
)const response = await client.connector.updateConfiguration({
connector_id: "my-spo-connector",
values: {
tenant_id: "my-tenant-id",
tenant_name: "my-sharepoint-site",
client_id: "foo",
secret_value: "bar",
site_collections: "*",
},
});response = client.connector.update_configuration(
connector_id: "my-spo-connector",
body: {
"values": {
"tenant_id": "my-tenant-id",
"tenant_name": "my-sharepoint-site",
"client_id": "foo",
"secret_value": "bar",
"site_collections": "*"
}
}
)$resp = $client->connector()->updateConfiguration([
"connector_id" => "my-spo-connector",
"body" => [
"values" => [
"tenant_id" => "my-tenant-id",
"tenant_name" => "my-sharepoint-site",
"client_id" => "foo",
"secret_value" => "bar",
"site_collections" => "*",
],
],
]);curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"values":{"tenant_id":"my-tenant-id","tenant_name":"my-sharepoint-site","client_id":"foo","secret_value":"bar","site_collections":"*"}}' "$ELASTICSEARCH_URL/_connector/my-spo-connector/_configuration"client.connector().updateConfiguration(u -> u
.connectorId("my-spo-connector")
.values(Map.of("tenant_id", JsonData.fromJson("\"my-tenant-id\""),"tenant_name", JsonData.fromJson("\"my-sharepoint-site\""),"secret_value", JsonData.fromJson("\"bar\""),"client_id", JsonData.fromJson("\"foo\""),"site_collections", JsonData.fromJson("\"*\"")))
);
{
"values": {
"tenant_id": "my-tenant-id",
"tenant_name": "my-sharepoint-site",
"client_id": "foo",
"secret_value": "bar",
"site_collections": "*"
}
}{
"values": {
"secret_value": "foo-bar"
}
}{
"result": "updated"
}Set the error field for the connector. If the error provided in the request body is non-null, the connector’s status is updated to error. Otherwise, if the error is reset to null, the connector status is updated to connected.
PUT _connector/my-connector/_error
{
"error": "Houston, we have a problem!"
}resp = client.connector.update_error(
connector_id="my-connector",
error="Houston, we have a problem!",
)const response = await client.connector.updateError({
connector_id: "my-connector",
error: "Houston, we have a problem!",
});response = client.connector.update_error(
connector_id: "my-connector",
body: {
"error": "Houston, we have a problem!"
}
)$resp = $client->connector()->updateError([
"connector_id" => "my-connector",
"body" => [
"error" => "Houston, we have a problem!",
],
]);curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"error":"Houston, we have a problem!"}' "$ELASTICSEARCH_URL/_connector/my-connector/_error"client.connector().updateError(u -> u
.connectorId("my-connector")
.error("Houston, we have a problem!")
);
{
"error": "Houston, we have a problem!"
}{
"result": "updated"
}Update the draft filtering configuration of a connector and marks the draft validation state as edited. The filtering draft is activated once validated by the running Elastic connector service. The filtering property is used to configure sync rules (both basic and advanced) for a connector.
PUT _connector/my-g-drive-connector/_filtering
{
"rules": [
{
"field": "file_extension",
"id": "exclude-txt-files",
"order": 0,
"policy": "exclude",
"rule": "equals",
"value": "txt"
},
{
"field": "_",
"id": "DEFAULT",
"order": 1,
"policy": "include",
"rule": "regex",
"value": ".*"
}
]
}resp = client.connector.update_filtering(
connector_id="my-g-drive-connector",
rules=[
{
"field": "file_extension",
"id": "exclude-txt-files",
"order": 0,
"policy": "exclude",
"rule": "equals",
"value": "txt"
},
{
"field": "_",
"id": "DEFAULT",
"order": 1,
"policy": "include",
"rule": "regex",
"value": ".*"
}
],
)const response = await client.connector.updateFiltering({
connector_id: "my-g-drive-connector",
rules: [
{
field: "file_extension",
id: "exclude-txt-files",
order: 0,
policy: "exclude",
rule: "equals",
value: "txt",
},
{
field: "_",
id: "DEFAULT",
order: 1,
policy: "include",
rule: "regex",
value: ".*",
},
],
});response = client.connector.update_filtering(
connector_id: "my-g-drive-connector",
body: {
"rules": [
{
"field": "file_extension",
"id": "exclude-txt-files",
"order": 0,
"policy": "exclude",
"rule": "equals",
"value": "txt"
},
{
"field": "_",
"id": "DEFAULT",
"order": 1,
"policy": "include",
"rule": "regex",
"value": ".*"
}
]
}
)$resp = $client->connector()->updateFiltering([
"connector_id" => "my-g-drive-connector",
"body" => [
"rules" => array(
[
"field" => "file_extension",
"id" => "exclude-txt-files",
"order" => 0,
"policy" => "exclude",
"rule" => "equals",
"value" => "txt",
],
[
"field" => "_",
"id" => "DEFAULT",
"order" => 1,
"policy" => "include",
"rule" => "regex",
"value" => ".*",
],
),
],
]);curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"rules":[{"field":"file_extension","id":"exclude-txt-files","order":0,"policy":"exclude","rule":"equals","value":"txt"},{"field":"_","id":"DEFAULT","order":1,"policy":"include","rule":"regex","value":".*"}]}' "$ELASTICSEARCH_URL/_connector/my-g-drive-connector/_filtering"client.connector().updateFiltering(u -> u
.connectorId("my-g-drive-connector")
.rules(List.of(FilteringRule.of(f -> f
.field("file_extension")
.id("exclude-txt-files")
.order(0)
.policy(FilteringPolicy.Exclude)
.rule(FilteringRuleRule.Equals)
.value("txt")),FilteringRule.of(f -> f
.field("_")
.id("DEFAULT")
.order(1)
.policy(FilteringPolicy.Include)
.rule(FilteringRuleRule.Regex)
.value(".*"))))
);
{
"rules": [
{
"field": "file_extension",
"id": "exclude-txt-files",
"order": 0,
"policy": "exclude",
"rule": "equals",
"value": "txt"
},
{
"field": "_",
"id": "DEFAULT",
"order": 1,
"policy": "include",
"rule": "regex",
"value": ".*"
}
]
}{
"advanced_snippet": {
"value": [{
"tables": [
"users",
"orders"
],
"query": "SELECT users.id AS id, orders.order_id AS order_id FROM users JOIN orders ON users.id = orders.user_id"
}]
}
}{
"result": "updated"
}PUT _connector/my-connector/_name
{
"name": "Custom connector",
"description": "This is my customized connector"
}resp = client.connector.update_name(
connector_id="my-connector",
name="Custom connector",
description="This is my customized connector",
)const response = await client.connector.updateName({
connector_id: "my-connector",
name: "Custom connector",
description: "This is my customized connector",
});response = client.connector.update_name(
connector_id: "my-connector",
body: {
"name": "Custom connector",
"description": "This is my customized connector"
}
)$resp = $client->connector()->updateName([
"connector_id" => "my-connector",
"body" => [
"name" => "Custom connector",
"description" => "This is my customized connector",
],
]);curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"name":"Custom connector","description":"This is my customized connector"}' "$ELASTICSEARCH_URL/_connector/my-connector/_name"client.connector().updateName(u -> u
.connectorId("my-connector")
.description("This is my customized connector")
.name("Custom connector")
);
{
"name": "Custom connector",
"description": "This is my customized connector"
}{
"result": "updated"
}When you create a new connector, the configuration of an ingest pipeline is populated with default settings.
PUT _connector/my-connector/_pipeline
{
"pipeline": {
"extract_binary_content": true,
"name": "my-connector-pipeline",
"reduce_whitespace": true,
"run_ml_inference": true
}
}resp = client.connector.update_pipeline(
connector_id="my-connector",
pipeline={
"extract_binary_content": True,
"name": "my-connector-pipeline",
"reduce_whitespace": True,
"run_ml_inference": True
},
)const response = await client.connector.updatePipeline({
connector_id: "my-connector",
pipeline: {
extract_binary_content: true,
name: "my-connector-pipeline",
reduce_whitespace: true,
run_ml_inference: true,
},
});response = client.connector.update_pipeline(
connector_id: "my-connector",
body: {
"pipeline": {
"extract_binary_content": true,
"name": "my-connector-pipeline",
"reduce_whitespace": true,
"run_ml_inference": true
}
}
)$resp = $client->connector()->updatePipeline([
"connector_id" => "my-connector",
"body" => [
"pipeline" => [
"extract_binary_content" => true,
"name" => "my-connector-pipeline",
"reduce_whitespace" => true,
"run_ml_inference" => true,
],
],
]);curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"pipeline":{"extract_binary_content":true,"name":"my-connector-pipeline","reduce_whitespace":true,"run_ml_inference":true}}' "$ELASTICSEARCH_URL/_connector/my-connector/_pipeline"client.connector().updatePipeline(u -> u
.connectorId("my-connector")
.pipeline(p -> p
.extractBinaryContent(true)
.name("my-connector-pipeline")
.reduceWhitespace(true)
.runMlInference(true)
)
);
{
"pipeline": {
"extract_binary_content": true,
"name": "my-connector-pipeline",
"reduce_whitespace": true,
"run_ml_inference": true
}
}{
"result": "updated"
}Remove the cross-cluster replication follower retention leases from the leader.
A following index takes out retention leases on its leader index. These leases are used to increase the likelihood that the shards of the leader index retain the history of operations that the shards of the following index need to run replication. When a follower index is converted to a regular index by the unfollow API (either by directly calling the API or by index lifecycle management tasks), these leases are removed. However, removal of the leases can fail, for example when the remote cluster containing the leader index is unavailable. While the leases will eventually expire on their own, their extended existence can cause the leader index to hold more history than necessary and prevent index lifecycle management from performing some operations on the leader index. This API exists to enable manually removing the leases when the unfollow API is unable to do so.
NOTE: This API does not stop replication by a following index. If you use this API with a follower index that is still actively following, the following index will add back retention leases on the leader. The only purpose of this API is to handle the case of failure to remove the following retention leases after the unfollow API is invoked.
the name of the leader index for which specified follower retention leases should be removed
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Values are -1 or 0.
POST /<leader_index>/_ccr/forget_follower
{
"follower_cluster" : "<follower_cluster>",
"follower_index" : "<follower_index>",
"follower_index_uuid" : "<follower_index_uuid>",
"leader_remote_cluster" : "<leader_remote_cluster>"
}resp = client.ccr.forget_follower(
index="<leader_index>",
follower_cluster="<follower_cluster>",
follower_index="<follower_index>",
follower_index_uuid="<follower_index_uuid>",
leader_remote_cluster="<leader_remote_cluster>",
)const response = await client.ccr.forgetFollower({
index: "<leader_index>",
follower_cluster: "<follower_cluster>",
follower_index: "<follower_index>",
follower_index_uuid: "<follower_index_uuid>",
leader_remote_cluster: "<leader_remote_cluster>",
});response = client.ccr.forget_follower(
index: "<leader_index>",
body: {
"follower_cluster": "<follower_cluster>",
"follower_index": "<follower_index>",
"follower_index_uuid": "<follower_index_uuid>",
"leader_remote_cluster": "<leader_remote_cluster>"
}
)$resp = $client->ccr()->forgetFollower([
"index" => "<leader_index>",
"body" => [
"follower_cluster" => "<follower_cluster>",
"follower_index" => "<follower_index>",
"follower_index_uuid" => "<follower_index_uuid>",
"leader_remote_cluster" => "<leader_remote_cluster>",
],
]);curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"follower_cluster":"<follower_cluster>","follower_index":"<follower_index>","follower_index_uuid":"<follower_index_uuid>","leader_remote_cluster":"<leader_remote_cluster>"}' "$ELASTICSEARCH_URL/<leader_index>/_ccr/forget_follower"client.ccr().forgetFollower(f -> f
.followerCluster("<follower_cluster>")
.followerIndex("<follower_index>")
.followerIndexUuid("<follower_index_uuid>")
.index("<leader_index>")
.leaderRemoteCluster("<leader_remote_cluster>")
);
{
"follower_cluster" : "<follower_cluster>",
"follower_index" : "<follower_index>",
"follower_index_uuid" : "<follower_index_uuid>",
"leader_remote_cluster" : "<leader_remote_cluster>"
}{
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0,
"failures" : [ ]
}
}Name of the data stream, which must meet the following criteria:
Lowercase only;
Cannot include \, /, *, ?, ", <, >, |, ,, #, :, or a space character;
Cannot start with -, _, +, or .ds-;
Cannot be . or ..;
Cannot be longer than 255 bytes. Multi-byte characters count towards this limit faster.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are -1 or 0.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Values are -1 or 0.
PUT _data_stream/logs-foo-bar
resp = client.indices.create_data_stream(
name="logs-foo-bar",
)const response = await client.indices.createDataStream({
name: "logs-foo-bar",
});response = client.indices.create_data_stream(
name: "logs-foo-bar"
)$resp = $client->indices()->createDataStream([
"name" => "logs-foo-bar",
]);curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_data_stream/logs-foo-bar"client.indices().createDataStream(c -> c
.name("logs-foo-bar")
);
Comma-separated list of data streams used to limit the request.
Wildcard expressions (*) are supported.
To target all data streams in a cluster, omit this parameter or use *.
Type of data stream that wildcard patterns can match.
Supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Values are all, open, closed, hidden, or none.
GET /_data_stream/my-index-000001/_stats
resp = client.indices.data_streams_stats(
name="my-index-000001",
)const response = await client.indices.dataStreamsStats({
name: "my-index-000001",
});response = client.indices.data_streams_stats(
name: "my-index-000001"
)$resp = $client->indices()->dataStreamsStats([
"name" => "my-index-000001",
]);curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_data_stream/my-index-000001/_stats"client.indices().dataStreamsStats(d -> d
.name("my-index-000001")
);
{
"_shards": {
"total": 10,
"successful": 5,
"failed": 0
},
"data_stream_count": 2,
"backing_indices": 5,
"total_store_size": "7kb",
"total_store_size_bytes": 7268,
"data_streams": [
{
"data_stream": "my-data-stream",
"backing_indices": 3,
"store_size": "3.7kb",
"store_size_bytes": 3772,
"maximum_timestamp": 1607512028000
},
{
"data_stream": "my-data-stream-two",
"backing_indices": 2,
"store_size": "3.4kb",
"store_size_bytes": 3496,
"maximum_timestamp": 1607425567000
}
]
}Get the data stream lifecycle configuration of one or more data streams.
Comma-separated list of data streams to limit the request.
Supports wildcards (*).
To target all data streams, omit this parameter or use * or _all.
Type of data stream that wildcard patterns can match.
Supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Values are all, open, closed, hidden, or none.
If true, return all default settings in the response.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Values are -1 or 0.
GET /_data_stream/{name}/_lifecycle?human&pretty
resp = client.indices.get_data_lifecycle(
name="{name}",
human=True,
pretty=True,
)const response = await client.indices.getDataLifecycle({
name: "{name}",
human: "true",
pretty: "true",
});response = client.indices.get_data_lifecycle(
name: "{name}",
human: "true",
pretty: "true"
)$resp = $client->indices()->getDataLifecycle([
"name" => "{name}",
"human" => "true",
"pretty" => "true",
]);curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_data_stream/%7Bname%7D/_lifecycle?human&pretty"{
"data_streams": [
{
"name": "my-data-stream-1",
"lifecycle": {
"enabled": true,
"data_retention": "7d"
}
},
{
"name": "my-data-stream-2",
"lifecycle": {
"enabled": true,
"data_retention": "7d"
}
}
]
}Aggregate a time series (TSDS) index and store pre-computed statistical summaries (min, max, sum, value_count and avg) for each metric field grouped by a configured time interval.
For example, a TSDS index that contains metrics sampled every 10 seconds can be downsampled to an hourly index.
All documents within an hour interval are summarized and stored as a single document in the downsample index.
NOTE: Only indices in a time series data stream are supported.
Neither field nor document level security can be defined on the source index.
The source index must be read only (index.blocks.write: true).
POST /my-time-series-index/_downsample/my-downsampled-time-series-index
{
"fixed_interval": "1d"
}resp = client.indices.downsample(
index="my-time-series-index",
target_index="my-downsampled-time-series-index",
config={
"fixed_interval": "1d"
},
)const response = await client.indices.downsample({
index: "my-time-series-index",
target_index: "my-downsampled-time-series-index",
config: {
fixed_interval: "1d",
},
});response = client.indices.downsample(
index: "my-time-series-index",
target_index: "my-downsampled-time-series-index",
body: {
"fixed_interval": "1d"
}
)$resp = $client->indices()->downsample([
"index" => "my-time-series-index",
"target_index" => "my-downsampled-time-series-index",
"body" => [
"fixed_interval" => "1d",
],
]);curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"fixed_interval":"1d"}' "$ELASTICSEARCH_URL/my-time-series-index/_downsample/my-downsampled-time-series-index"client.indices().downsample(d -> d
.index("my-time-series-index")
.targetIndex("my-downsampled-time-series-index")
.config(c -> c
.fixedInterval(f -> f
.time("1d")
)
)
);
{
"fixed_interval": "1d"
}Get information about an index or data stream's current data stream lifecycle status, such as time since index creation, time since rollover, the lifecycle configuration managing the index, or any errors encountered during lifecycle execution.
GET .ds-metrics-2023.03.22-000001/_lifecycle/explain
resp = client.indices.explain_data_lifecycle(
index=".ds-metrics-2023.03.22-000001",
)const response = await client.indices.explainDataLifecycle({
index: ".ds-metrics-2023.03.22-000001",
});response = client.indices.explain_data_lifecycle(
index: ".ds-metrics-2023.03.22-000001"
)$resp = $client->indices()->explainDataLifecycle([
"index" => ".ds-metrics-2023.03.22-000001",
]);curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/.ds-metrics-2023.03.22-000001/_lifecycle/explain"client.indices().explainDataLifecycle(e -> e
.index(".ds-metrics-2023.03.22-000001")
);
{
"indices": {
".ds-metrics-2023.03.22-000001": {
"index" : ".ds-metrics-2023.03.22-000001",
"managed_by_lifecycle" : true,
"index_creation_date_millis" : 1679475563571,
"time_since_index_creation" : "843ms",
"rollover_date_millis" : 1679475564293,
"time_since_rollover" : "121ms",
"lifecycle" : { },
"generation_time" : "121ms"
}
}{
"indices": {
".ds-metrics-2023.03.22-000001": {
"index" : ".ds-metrics-2023.03.22-000001",
"managed_by_lifecycle" : true,
"index_creation_date_millis" : 1679475563571,
"time_since_index_creation" : "843ms",
"lifecycle" : {
"enabled": true
},
"error": "{\"type\":\"validation_exception\",\"reason\":\"Validation Failed: 1: this action would add [2] shards, but this cluster
currently has [4]/[3] maximum normal shards open;\"}"
}
}Performs one or more data stream modification actions in a single atomic operation.
POST _data_stream/_modify
{
"actions": [
{
"remove_backing_index": {
"data_stream": "my-data-stream",
"index": ".ds-my-data-stream-2023.07.26-000001"
}
},
{
"add_backing_index": {
"data_stream": "my-data-stream",
"index": ".ds-my-data-stream-2023.07.26-000001-downsample"
}
}
]
}resp = client.indices.modify_data_stream(
actions=[
{
"remove_backing_index": {
"data_stream": "my-data-stream",
"index": ".ds-my-data-stream-2023.07.26-000001"
}
},
{
"add_backing_index": {
"data_stream": "my-data-stream",
"index": ".ds-my-data-stream-2023.07.26-000001-downsample"
}
}
],
)const response = await client.indices.modifyDataStream({
actions: [
{
remove_backing_index: {
data_stream: "my-data-stream",
index: ".ds-my-data-stream-2023.07.26-000001",
},
},
{
add_backing_index: {
data_stream: "my-data-stream",
index: ".ds-my-data-stream-2023.07.26-000001-downsample",
},
},
],
});response = client.indices.modify_data_stream(
body: {
"actions": [
{
"remove_backing_index": {
"data_stream": "my-data-stream",
"index": ".ds-my-data-stream-2023.07.26-000001"
}
},
{
"add_backing_index": {
"data_stream": "my-data-stream",
"index": ".ds-my-data-stream-2023.07.26-000001-downsample"
}
}
]
}
)$resp = $client->indices()->modifyDataStream([
"body" => [
"actions" => array(
[
"remove_backing_index" => [
"data_stream" => "my-data-stream",
"index" => ".ds-my-data-stream-2023.07.26-000001",
],
],
[
"add_backing_index" => [
"data_stream" => "my-data-stream",
"index" => ".ds-my-data-stream-2023.07.26-000001-downsample",
],
],
),
],
]);curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"actions":[{"remove_backing_index":{"data_stream":"my-data-stream","index":".ds-my-data-stream-2023.07.26-000001"}},{"add_backing_index":{"data_stream":"my-data-stream","index":".ds-my-data-stream-2023.07.26-000001-downsample"}}]}' "$ELASTICSEARCH_URL/_data_stream/_modify"client.indices().modifyDataStream(m -> m
.actions(List.of(Action.of(a -> a
.removeBackingIndex(r -> r
.dataStream("my-data-stream")
.index(".ds-my-data-stream-2023.07.26-000001")
)),Action.of(ac -> ac
.addBackingIndex(ad -> ad
.dataStream("my-data-stream")
.index(".ds-my-data-stream-2023.07.26-000001-downsample")
))))
);
{
"actions": [
{
"remove_backing_index": {
"data_stream": "my-data-stream",
"index": ".ds-my-data-stream-2023.07.26-000001"
}
},
{
"add_backing_index": {
"data_stream": "my-data-stream",
"index": ".ds-my-data-stream-2023.07.26-000001-downsample"
}
}
]
}All methods and paths for this operation:
You can index a new JSON document with the /<target>/_doc/ or /<target>/_create/<_id> APIs
Using _create guarantees that the document is indexed only if it does not already exist.
It returns a 409 response when a document with a same ID already exists in the index.
To update an existing document, you must use the /<target>/_doc/ API.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
PUT /<target>/_create/<_id> or POST /<target>/_create/<_id> request formats, you must have the create_doc, create, index, or write index privilege.auto_configure, create_index, or manage index privilege.Automatic data stream creation requires a matching index template with data stream enabled.
Automatically create data streams and indices
If the request's target doesn't exist and matches an index template with a data_stream definition, the index operation automatically creates the data stream.
If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.
NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.
If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.
Automatic index creation is controlled by the action.auto_create_index setting.
If it is true, any index can be created automatically.
You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false to turn off automatic index creation entirely.
Specify a comma-separated list of patterns you want to allow or prefix each pattern with + or - to indicate whether it should be allowed or blocked.
When a list is specified, the default behaviour is to disallow.
NOTE: The action.auto_create_index setting affects the automatic creation of indices only.
It does not affect the creation of data streams.
Routing
By default, shard placement — or routing — is controlled by using a hash of the document's ID value.
For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing parameter.
When setting up explicit mapping, you can also use the _routing field to direct the index operation to extract the routing value from the document itself.
This does come at the (very minimal) cost of an additional document parsing pass.
If the _routing mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.
Distributed
The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.
Active shards
To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation.
If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs.
By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards is 1).
This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards.
To alter this behavior per operation, use the wait_for_active_shards request parameter.
Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas+1).
Specifying a negative value or a number greater than the number of shard copies will throw an error.
For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes).
If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding.
This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data.
If wait_for_active_shards is set on the request to 3 (and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding.
This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard.
However, if you set wait_for_active_shards to all (or to 4, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index.
The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.
It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts.
After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary.
The _shards section of the API response reveals the number of shard copies on which replication succeeded and failed.
createThe name of the data stream or index to target.
If the target doesn't exist and matches the name or wildcard (*) pattern of an index template with a data_stream definition, this request creates the data stream.
If the target doesn't exist and doesn’t match a data stream template, this request creates the index.
A unique identifier for the document.
To automatically generate a document ID, use the POST /<target>/_doc/ request format.
True or false if to include the document source in the error message in case of parsing errors.
The ID of the pipeline to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request.
If a final pipeline is configured, it will always run regardless of the value of this parameter.
If true, Elasticsearch refreshes the affected shards to make this operation visible to search.
If wait_for, it waits for a refresh to make this operation visible to search.
If false, it does nothing with refreshes.
Values are true, false, or wait_for.
If true, the destination must be an index alias.
If true, the request's actions must target a data stream (existing or to be created).
A custom value that is used to route operations to a specific shard.
The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards. Elasticsearch waits for at least the specified timeout period before failing. The actual wait time could be longer, particularly when multiple waits occur.
This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.
Values are -1 or 0.
The explicit version number for concurrency control. It must be a non-negative long number.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
The number of shard copies that must be active before proceeding with the operation.
You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default value of 1 means it waits for each primary shard to be active.
Values are all or index-setting.
PUT my-index-000001/_create/1
{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}resp = client.create(
index="my-index-000001",
id="1",
document={
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
},
)const response = await client.create({
index: "my-index-000001",
id: 1,
document: {
"@timestamp": "2099-11-15T13:12:00",
message: "GET /search HTTP/1.1 200 1070000",
user: {
id: "kimchy",
},
},
});response = client.create(
index: "my-index-000001",
id: "1",
body: {
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}
)$resp = $client->create([
"index" => "my-index-000001",
"id" => "1",
"body" => [
"@timestamp" => "2099-11-15T13:12:00",
"message" => "GET /search HTTP/1.1 200 1070000",
"user" => [
"id" => "kimchy",
],
],
]);curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"@timestamp":"2099-11-15T13:12:00","message":"GET /search HTTP/1.1 200 1070000","user":{"id":"kimchy"}}' "$ELASTICSEARCH_URL/my-index-000001/_create/1"client.create(c -> c
.id("1")
.index("my-index-000001")
.document(JsonData.fromJson("{\"@timestamp\":\"2099-11-15T13:12:00\",\"message\":\"GET /search HTTP/1.1 200 1070000\",\"user\":{\"id\":\"kimchy\"}}"))
);
{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}{
"_index": "my-index-000001",
"_id": "1",
"_version": 1,
"result": "created",
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"_seq_no": 0,
"_primary_term": 1
}Get a document and its source or stored fields from an index.
By default, this API is realtime and is not affected by the refresh rate of the index (when data will become visible for search).
In the case where stored fields are requested with the stored_fields parameter and the document has been updated but is not yet refreshed, the API will have to parse and analyze the source to extract the stored fields.
To turn off realtime behavior, set the realtime parameter to false.
Source filtering
By default, the API returns the contents of the _source field unless you have used the stored_fields parameter or the _source field is turned off.
You can turn off _source retrieval by using the _source parameter:
GET my-index-000001/_doc/0?_source=false
If you only need one or two fields from the _source, use the _source_includes or _source_excludes parameters to include or filter out particular fields.
This can be helpful with large documents where partial retrieval can save on network overhead
Both parameters take a comma separated list of fields or wildcard expressions.
For example:
GET my-index-000001/_doc/0?_source_includes=*.id&_source_excludes=entities
If you only want to specify includes, you can use a shorter notation:
GET my-index-000001/_doc/0?_source=*.id
Routing
If routing is used during indexing, the routing value also needs to be specified to retrieve a document. For example:
GET my-index-000001/_doc/2?routing=user1
This request gets the document with ID 2, but it is routed based on the user. The document is not fetched if the correct routing is not specified.
Distributed
The GET operation is hashed into a specific shard ID. It is then redirected to one of the replicas within that shard ID and returns the result. The replicas are the primary shard and its replicas within that shard ID group. This means that the more replicas you have, the better your GET scaling will be.
Versioning support
You can use the version parameter to retrieve the document only if its current version is equal to the specified one.
Internally, Elasticsearch has marked the old document as deleted and added an entirely new document. The old version of the document doesn't disappear immediately, although you won't be able to access it. Elasticsearch cleans up deleted documents in the background as you continue to index more data.
readThe node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.
If it is set to _local, the operation will prefer to be run on a local allocated shard when possible.
If it is set to a custom value, the value is used to guarantee that the same shards will be used for the same custom value.
This can help with "jumping values" when hitting different shards in different refresh states.
A sample value can be something like the web session ID or the user name.
If true, the request is real-time as opposed to near-real-time.
If true, the request refreshes the relevant shards before retrieving the document.
Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).
A custom value used to route operations to a specific shard.
Indicates whether to return the _source field (true or false) or lists the fields to return.
A comma-separated list of source fields to exclude from the response.
You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.
If the _source parameter is false, this parameter is ignored.
Whether vectors should be excluded from _source
A comma-separated list of source fields to include in the response.
If this parameter is specified, only these source fields are returned.
You can exclude fields from this subset using the _source_excludes query parameter.
If the _source parameter is false, this parameter is ignored.
A comma-separated list of stored fields to return as part of a hit.
If no fields are specified, no stored fields are included in the response.
If this field is specified, the _source parameter defaults to false.
Only leaf fields can be retrieved with the stored_fields option.
Object fields can't be returned; if specified, the request fails.
The version number for concurrency control. It must match the current version of the document for the request to succeed.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
GET my-index-000001/_doc/1?stored_fields=tags,counter
resp = client.get(
index="my-index-000001",
id="1",
stored_fields="tags,counter",
)const response = await client.get({
index: "my-index-000001",
id: 1,
stored_fields: "tags,counter",
});response = client.get(
index: "my-index-000001",
id: "1",
stored_fields: "tags,counter"
)$resp = $client->get([
"index" => "my-index-000001",
"id" => "1",
"stored_fields" => "tags,counter",
]);curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/my-index-000001/_doc/1?stored_fields=tags,counter"{
"_index": "my-index-000001",
"_id": "0",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"found": true,
"_source": {
"@timestamp": "2099-11-15T14:12:12",
"http": {
"request": {
"method": "get"
},
"response": {
"status_code": 200,
"bytes": 1070000
},
"version": "1.1"
},
"source": {
"ip": "127.0.0.1"
},
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}
}{
"_index": "my-index-000001",
"_id": "1",
"_version": 1,
"_seq_no" : 22,
"_primary_term" : 1,
"found": true,
"fields": {
"tags": [
"production"
]
}
}{
"_index": "my-index-000001",
"_id": "2",
"_version": 1,
"_seq_no" : 13,
"_primary_term" : 1,
"_routing": "user1",
"found": true,
"fields": {
"tags": [
"env2"
]
}
}Check whether a document source exists in an index. For example:
HEAD my-index-000001/_source/1
A document's source is not available if it is disabled in the mapping.
readA comma-separated list of data streams, indices, and aliases.
It supports wildcards (*).
A unique identifier for the document.
The node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.
If true, the request is real-time as opposed to near-real-time.
If true, the request refreshes the relevant shards before retrieving the document.
Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).
A custom value used to route operations to a specific shard.
Indicates whether to return the _source field (true or false) or lists the fields to return.
A comma-separated list of source fields to exclude in the response.
A comma-separated list of source fields to include in the response.
The version number for concurrency control. It must match the current version of the document for the request to succeed.
The version type.
Supported values include:
internal: Use internal versioning that starts at 1 and increments with each update or delete.external: Only index the document if the specified version is strictly higher than the version of the stored document or if there is no existing document.external_gte: Only index the document if the specified version is equal or higher than the version of the stored document or if there is no existing document.
NOTE: The external_gte version type is meant for special use cases and should be used with care.
If used incorrectly, it can result in loss of data.force: This option is deprecated because it can cause primary and replica shards to diverge.Values are internal, external, external_gte, or force.
HEAD my-index-000001/_source/1
resp = client.exists_source(
index="my-index-000001",
id="1",
)const response = await client.existsSource({
index: "my-index-000001",
id: 1,
});response = client.exists_source(
index: "my-index-000001",
id: "1"
)$resp = $client->existsSource([
"index" => "my-index-000001",
"id" => "1",
]);curl --head -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/my-index-000001/_source/1"client.existsSource(e -> e
.id("1")
.index("my-index-000001")
);
Updates documents that match the specified query. If no query is specified, performs an update on every document in the data stream or index without modifying the source, which is useful for picking up mapping changes.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or alias:
readindex or writeYou can specify the query criteria in the request URI or the request body using the same syntax as the search API.
When you submit an update by query request, Elasticsearch gets a snapshot of the data stream or index when it begins processing the request and updates matching documents using internal versioning.
When the versions match, the document is updated and the version number is incremented.
If a document changes between the time that the snapshot is taken and the update operation is processed, it results in a version conflict and the operation fails.
You can opt to count version conflicts instead of halting and returning by setting conflicts to proceed.
Note that if you opt to count version conflicts, the operation could attempt to update more documents from the source than max_docs until it has successfully updated max_docs documents or it has gone through every document in the source query.
NOTE: Documents with a version equal to 0 cannot be updated using update by query because internal versioning does not support 0 as a valid version number.
While processing an update by query request, Elasticsearch performs multiple search requests sequentially to find all of the matching documents. A bulk update request is performed for each batch of matching documents. Any query or update failures cause the update by query request to fail and the failures are shown in the response. Any update requests that completed successfully still stick, they are not rolled back.
Refreshing shards
Specifying the refresh parameter refreshes all shards once the request completes.
This is different to the update API's refresh parameter, which causes only the shard
that received the request to be refreshed. Unlike the update API, it does not support
wait_for.
Running update by query asynchronously
If the request contains wait_for_completion=false, Elasticsearch
performs some preflight checks, launches the request, and returns a
task you can use to cancel or get the status of the task.
Elasticsearch creates a record of this task as a document at .tasks/task/${taskId}.
Waiting for active shards
wait_for_active_shards controls how many copies of a shard must be active
before proceeding with the request. See wait_for_active_shards
for details. timeout controls how long each write request waits for unavailable
shards to become available. Both work exactly the way they work in the
Bulk API. Update by query uses scrolled searches, so you can also
specify the scroll parameter to control how long it keeps the search context
alive, for example ?scroll=10m. The default is 5 minutes.
Throttling update requests
To control the rate at which update by query issues batches of update operations, you can set requests_per_second to any positive decimal number.
This pads each batch with a wait time to throttle the rate.
Set requests_per_second to -1 to turn off throttling.
Throttling uses a wait time between batches so that the internal scroll requests can be given a timeout that takes the request padding into account.
The padding time is the difference between the batch size divided by the requests_per_second and the time spent writing.
By default the batch size is 1000, so if requests_per_second is set to 500:
target_time = 1000 / 500 per second = 2 seconds
wait_time = target_time - write_time = 2 seconds - .5 seconds = 1.5 seconds
Since the batch is issued as a single _bulk request, large batch sizes cause Elasticsearch to create many requests and wait before starting the next set. This is "bursty" instead of "smooth".
Slicing
Update by query supports sliced scroll to parallelize the update process. This can improve efficiency and provide a convenient way to break the request down into smaller parts.
Setting slices to auto chooses a reasonable number for most data streams and indices.
This setting will use one slice per shard, up to a certain limit.
If there are multiple source data streams or indices, it will choose the number of slices based on the index or backing index with the smallest number of shards.
Adding slices to _update_by_query just automates the manual process of creating sub-requests, which means it has some quirks:
slices only contains the status of completed slices.slices will rethrottle the unfinished sub-request proportionally.requests_per_second and max_docs on a request with slices are distributed proportionally to each sub-request. Combine that with the point above about distribution being uneven and you should conclude that using max_docs with slices might not result in exactly max_docs documents being updated.If you're slicing manually or otherwise tuning automatic slicing, keep in mind that:
Whether query or update performance dominates the runtime depends on the documents being reindexed and cluster resources.
Refer to the linked documentation for examples of how to update documents using the _update_by_query API:
read,writeA comma-separated list of data streams, indices, and aliases to search.
It supports wildcards (*).
To search all data streams or indices, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
The analyzer to use for the query string.
This parameter can be used only when the q query string parameter is specified.
If true, wildcard and prefix queries are analyzed.
This parameter can be used only when the q query string parameter is specified.
The preferred behavior when update by query hits version conflicts: abort or proceed.
Supported values include:
abort: Stop reindexing if there are conflicts.proceed: Continue reindexing even if there are conflicts.Values are abort or proceed.
The default operator for query string query: AND or OR.
This parameter can be used only when the q query string parameter is specified.
Values are and, AND, or, or OR.
The field to use as default where no field prefix is given in the query string.
This parameter can be used only when the q query string parameter is specified.
The type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
It supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Values are all, open, closed, hidden, or none.
Skips the specified number of documents.
If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored.
This parameter can be used only when the q query string parameter is specified.
The maximum number of documents to process.
It defaults to all documents.
When set to a value less then or equal to scroll_size then a scroll will not be used to retrieve the results for the operation.
The ID of the pipeline to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, then setting the value to _none disables the default ingest pipeline for this request.
If a final pipeline is configured it will always run, regardless of the value of this parameter.
The node or shard the operation should be performed on. It is random by default.
A query in the Lucene query string syntax.
If true, Elasticsearch refreshes affected shards to make the operation visible to search after the request completes.
This is different than the update API's refresh parameter, which causes just the shard that received the request to be refreshed.
If true, the request cache is used for this request.
It defaults to the index-level setting.
The throttle for this request in sub-requests per second.
A custom value used to route operations to a specific shard.
The period to retain the search context for scrolling.
Values are -1 or 0.
The size of the scroll request that powers the operation.
An explicit timeout for each search request. By default, there is no timeout.
Values are -1 or 0.
The type of the search operation. Available options include query_then_fetch and dfs_query_then_fetch.
Supported values include:
query_then_fetch: Documents are scored using local term and document frequencies for the shard. This is usually faster but less accurate.dfs_query_then_fetch: Documents are scored using global term and document frequencies across all shards. This is usually slower but more accurate.Values are query_then_fetch or dfs_query_then_fetch.
The number of slices this task should be divided into.
Value is auto.
A comma-separated list of : pairs.
The specific tag of the request for logging and statistical purposes.
The maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting.
IMPORTANT: Use with caution. Elasticsearch applies this parameter to each shard handling the request. When possible, let Elasticsearch perform early termination automatically. Avoid specifying this parameter for requests that target data streams with backing indices across multiple data tiers.
The period each update request waits for the following operations: dynamic mapping updates, waiting for active shards. By default, it is one minute. This guarantees Elasticsearch waits for at least the timeout before failing. The actual wait time could be longer, particularly when multiple waits occur.
Values are -1 or 0.
If true, returns the document version as part of a hit.
Should the document increment the version number (internal) on hit or not (reindex)
The number of shard copies that must be active before proceeding with the operation.
Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The timeout parameter controls how long each write request waits for unavailable shards to become available.
Both work exactly the way they work in the bulk API.
Values are all or index-setting.
If true, the request blocks until the operation is complete.
If false, Elasticsearch performs some preflight checks, launches the request, and returns a task ID that you can use to cancel or get the status of the task.
Elasticsearch creates a record of this task as a document at .tasks/task/${taskId}.
The maximum number of documents to update.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
Values are abort or proceed.
POST my-index-000001/_update_by_query?conflicts=proceed
{
"query": {
"term": {
"user.id": "kimchy"
}
}
}resp = client.update_by_query(
index="my-index-000001",
conflicts="proceed",
query={
"term": {
"user.id": "kimchy"
}
},
)const response = await client.updateByQuery({
index: "my-index-000001",
conflicts: "proceed",
query: {
term: {
"user.id": "kimchy",
},
},
});response = client.update_by_query(
index: "my-index-000001",
conflicts: "proceed",
body: {
"query": {
"term": {
"user.id": "kimchy"
}
}
}
)$resp = $client->updateByQuery([
"index" => "my-index-000001",
"conflicts" => "proceed",
"body" => [
"query" => [
"term" => [
"user.id" => "kimchy",
],
],
],
]);curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"query":{"term":{"user.id":"kimchy"}}}' "$ELASTICSEARCH_URL/my-index-000001/_update_by_query?conflicts=proceed"client.updateByQuery(u -> u
.conflicts(Conflicts.Proceed)
.index("my-index-000001")
.query(q -> q
.term(t -> t
.field("user.id")
.value(FieldValue.of("kimchy"))
)
)
);
{
"query": {
"term": {
"user.id": "kimchy"
}
}
}{
"script": {
"source": "ctx._source.count++",
"lang": "painless"
},
"query": {
"term": {
"user.id": "kimchy"
}
}
}{
"slice": {
"id": 0,
"max": 2
},
"script": {
"source": "ctx._source['extra'] = 'test'"
}
}{
"script": {
"source": "ctx._source['extra'] = 'test'"
}
}Delete an async EQL search or a stored synchronous EQL search. The API also deletes results for the search.
DELETE /_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=
resp = client.eql.delete(
id="FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
)const response = await client.eql.delete({
id: "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
});response = client.eql.delete(
id: "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE="
)$resp = $client->eql()->delete([
"id" => "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
]);curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE="client.eql().delete(d -> d
.id("FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=")
);
GET /_eql/search/status/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=
resp = client.eql.get_status(
id="FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
)const response = await client.eql.getStatus({
id: "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
});response = client.eql.get_status(
id: "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE="
)$resp = $client->eql()->getStatus([
"id" => "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
]);curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_eql/search/status/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE="client.eql().getStatus(g -> g
.id("FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=")
);
{
"id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
"is_running" : true,
"is_partial" : true,
"start_time_in_millis" : 1611690235000,
"expiration_time_in_millis" : 1611690295000
}All methods and paths for this operation:
Returns search results for an Event Query Language (EQL) query. EQL assumes each document in a data stream or index corresponds to an event.
Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)
If true, returns partial results if there are shard failures. If false, returns an error with no partial results.
If true, sequence queries will return partial results in case of shard failures. If false, they will return no results at all. This flag has effect only if allow_partial_search_results is true.
Whether to expand wildcard expression to concrete indices that are open, closed or both.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.none: Wildcard expressions are not accepted.Values are all, open, closed, hidden, or none.
Indicates whether network round-trips should be minimized as part of cross-cluster search requests execution
Period for which the search and its results are stored on the cluster.
Values are -1 or 0.
If true, the search and its results are stored on the cluster.
Timeout duration to wait for the request to finish. Defaults to no timeout, meaning the request waits for complete search results.
Values are -1 or 0.
EQL query you wish to run.
Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
Query, written in Query DSL, used to filter the events on which the EQL query runs.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
Allow query execution also in case of shard failures. If true, the query will keep running and will return results based on the available shards. For sequences, the behavior can be further refined using allow_partial_sequence_results
Default value is true.
This flag applies only to sequences and has effect only if allow_partial_search_results=true. If true, the sequence query will return results based on the available shards, ignoring the others. If false, the sequence query will return successfully, but will always have empty results.
Default value is false.
Values are tail or head.
By default, the response of a sample query contains up to 10 samples, with one sample per unique set of join keys. Use the size
parameter to get a smaller or larger set of samples. To retrieve more than one sample per set of join keys, use the
max_samples_per_key parameter. Pipes are not supported for sample queries.
Default value is 1.
GET /my-data-stream/_eql/search
{
"query": """
process where (process.name == "cmd.exe" and process.pid != 2013)
"""
}resp = client.eql.search(
index="my-data-stream",
query="\n process where (process.name == \"cmd.exe\" and process.pid != 2013)\n ",
)const response = await client.eql.search({
index: "my-data-stream",
query:
'\n process where (process.name == "cmd.exe" and process.pid != 2013)\n ',
});response = client.eql.search(
index: "my-data-stream",
body: {
"query": "\n process where (process.name == \"cmd.exe\" and process.pid != 2013)\n "
}
)$resp = $client->eql()->search([
"index" => "my-data-stream",
"body" => [
"query" => "\n process where (process.name == \"cmd.exe\" and process.pid != 2013)\n ",
],
]);curl -X GET -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"query":"\n process where (process.name == \"cmd.exe\" and process.pid != 2013)\n "}' "$ELASTICSEARCH_URL/my-data-stream/_eql/search"client.eql().search(s -> s
.index("my-data-stream")
.query(" process where (process.name == \"cmd.exe\" and process.pid != 2013) ")
);
{
"query": """
process where (process.name == "cmd.exe" and process.pid != 2013)
"""
}{
"query": """
sequence by process.pid
[ file where file.name == "cmd.exe" and process.pid != 2013 ]
[ process where stringContains(process.executable, "regsvr32") ]
"""
}{
"is_partial": false,
"is_running": false,
"took": 6,
"timed_out": false,
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"sequences": [
{
"join_keys": [
2012
],
"events": [
{
"_index": ".ds-my-data-stream-2099.12.07-000001",
"_id": "AtOJ4UjUBAAx3XR5kcCM",
"_source": {
"@timestamp": "2099-12-06T11:04:07.000Z",
"event": {
"category": "file",
"id": "dGCHwoeS",
"sequence": 2
},
"file": {
"accessed": "2099-12-07T11:07:08.000Z",
"name": "cmd.exe",
"path": "C:\\Windows\\System32\\cmd.exe",
"type": "file",
"size": 16384
},
"process": {
"pid": 2012,
"name": "cmd.exe",
"executable": "C:\\Windows\\System32\\cmd.exe"
}
}
},
{
"_index": ".ds-my-data-stream-2099.12.07-000001",
"_id": "OQmfCaduce8zoHT93o4H",
"_source": {
"@timestamp": "2099-12-07T11:07:09.000Z",
"event": {
"category": "process",
"id": "aR3NWVOs",
"sequence": 4
},
"process": {
"pid": 2012,
"name": "regsvr32.exe",
"command_line": "regsvr32.exe /s /u /i:https://...RegSvr32.sct scrobj.dll",
"executable": "C:\\Windows\\System32\\regsvr32.exe"
}
}
}
]
}
]
}
}If the query is still running, it is cancelled. Otherwise, the stored results are deleted.
If the Elasticsearch security features are enabled, only the following users can use this API to delete a query:
cancel_task cluster privilegeDELETE /_query/async/FmdMX2pIang3UWhLRU5QS0lqdlppYncaMUpYQ05oSkpTc3kwZ21EdC1tbFJXQToxOTI=
resp = client.esql.async_query_delete(
id="FmdMX2pIang3UWhLRU5QS0lqdlppYncaMUpYQ05oSkpTc3kwZ21EdC1tbFJXQToxOTI=",
)const response = await client.esql.asyncQueryDelete({
id: "FmdMX2pIang3UWhLRU5QS0lqdlppYncaMUpYQ05oSkpTc3kwZ21EdC1tbFJXQToxOTI=",
});response = client.esql.async_query_delete(
id: "FmdMX2pIang3UWhLRU5QS0lqdlppYncaMUpYQ05oSkpTc3kwZ21EdC1tbFJXQToxOTI="
)$resp = $client->esql()->asyncQueryDelete([
"id" => "FmdMX2pIang3UWhLRU5QS0lqdlppYncaMUpYQ05oSkpTc3kwZ21EdC1tbFJXQToxOTI=",
]);curl -X DELETE -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_query/async/FmdMX2pIang3UWhLRU5QS0lqdlppYncaMUpYQ05oSkpTc3kwZ21EdC1tbFJXQToxOTI="Get search results for an ES|QL (Elasticsearch query language) query.
A short version of the Accept header, e.g. json, yaml.
csv, tsv, and txt formats will return results in a tabular format, excluding other metadata fields from the response.
Values are csv, json, tsv, txt, yaml, cbor, smile, or arrow.
The character to use between values within a CSV row. Only valid for the CSV format.
Should columns that are entirely null be removed from the columns and values portion of the results?
Defaults to false. If true then the response will include an extra section under the name all_columns which has the name of all columns.
If true, partial results will be returned if there are shard failures, but the query can continue to execute on other clusters and shards.
If false, the query will fail if there are any failures.
To override the default behavior, you can set the esql.query.allow_partial_results cluster setting to false.
By default, ES|QL returns results as rows. For example, FROM returns each individual document as one row. For the JSON, YAML, CBOR and smile formats, ES|QL can return the results in a columnar fashion where one row represents all the values of a certain column in the results.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
To avoid any attempts of hacking or code injection, extract the values in a separate list of parameters. Use question mark placeholders (?) in the query string for each of the parameters.
If provided and true the response will include an extra profile object
with information on how the query was executed. This information is for human debugging
and its format can change at any time but it can give some insight into the performance
of each part of the query.
The ES|QL query API accepts an ES|QL query string in the query parameter, runs it, and returns the results.
Tables to use with the LOOKUP operation. The top level key is the table name and the next level key is the column name.
When set to true and performing a cross-cluster query, the response will include an extra _clusters
object with information about the clusters that participated in the search along with info such as shards
count.
Default value is false.
POST /_query
{
"query": """
FROM library,remote-*:library
| EVAL year = DATE_TRUNC(1 YEARS, release_date)
| STATS MAX(page_count) BY year
| SORT year
| LIMIT 5
""",
"include_ccs_metadata": true
}resp = client.esql.query(
query="\n FROM library,remote-*:library\n | EVAL year = DATE_TRUNC(1 YEARS, release_date)\n | STATS MAX(page_count) BY year\n | SORT year\n | LIMIT 5\n ",
include_ccs_metadata=True,
)const response = await client.esql.query({
query:
"\n FROM library,remote-*:library\n | EVAL year = DATE_TRUNC(1 YEARS, release_date)\n | STATS MAX(page_count) BY year\n | SORT year\n | LIMIT 5\n ",
include_ccs_metadata: true,
});response = client.esql.query(
body: {
"query": "\n FROM library,remote-*:library\n | EVAL year = DATE_TRUNC(1 YEARS, release_date)\n | STATS MAX(page_count) BY year\n | SORT year\n | LIMIT 5\n ",
"include_ccs_metadata": true
}
)$resp = $client->esql()->query([
"body" => [
"query" => "\n FROM library,remote-*:library\n | EVAL year = DATE_TRUNC(1 YEARS, release_date)\n | STATS MAX(page_count) BY year\n | SORT year\n | LIMIT 5\n ",
"include_ccs_metadata" => true,
],
]);curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"query":"\n FROM library,remote-*:library\n | EVAL year = DATE_TRUNC(1 YEARS, release_date)\n | STATS MAX(page_count) BY year\n | SORT year\n | LIMIT 5\n ","include_ccs_metadata":true}' "$ELASTICSEARCH_URL/_query"client.esql().query(q -> q
.includeCcsMetadata(true)
.query(" FROM library,remote-*:library | EVAL year = DATE_TRUNC(1 YEARS, release_date) | STATS MAX(page_count) BY year | SORT year | LIMIT 5 ")
);
{
"query": """
FROM library,remote-*:library
| EVAL year = DATE_TRUNC(1 YEARS, release_date)
| STATS MAX(page_count) BY year
| SORT year
| LIMIT 5
""",
"include_ccs_metadata": true
}Clear all of the state information stored in system indices by Elasticsearch features, including the security and machine learning indices.
WARNING: Intended for development and testing use only. Do not reset features on a production cluster.
Return a cluster to the same state as a new installation by resetting the feature state for all Elasticsearch features. This deletes all state information stored in system indices.
The response code is HTTP 200 if the state is successfully reset for all features. It is HTTP 500 if the reset operation failed for any feature.
Note that select features might provide a way to reset particular system indices. Using this API resets all features, both those that are built-in and implemented as plugins.
To list the features that will be affected, use the get features API.
IMPORTANT: The features installed on the node you submit this request to are the features that will be reset. Run on the master node if you have any doubts about which plugins are installed on individual nodes.
POST /_features/_reset
resp = client.features.reset_features()const response = await client.features.resetFeatures();response = client.features.reset_features$resp = $client->features()->resetFeatures();curl -X POST -H "Authorization: ApiKey $ELASTIC_API_KEY" "$ELASTICSEARCH_URL/_features/_reset"client.features().resetFeatures(r -> r);
{
"features" : [
{
"feature_name" : "security",
"status" : "SUCCESS"
},
{
"feature_name" : "tasks",
"status" : "SUCCESS"
}
]
}