Prepare to upgrade

To identify indices that require action, use the Deprecation info API:

GET /.ml-anomalies-*/_migration/deprecations

The response contains the list of critical deprecation warnings in the index_settings section:

"index_settings": {
  ".ml-anomalies-shared": [
    {
      "level": "critical",
      "message": "Index created before 8.0",
      "url": "https://ela.st/es-deprecation-8-reindex",
      "details": "This index was created with version 7.8.23 and is not compatible with 9.0. Reindex or remove the index before upgrading.",
      "resolve_during_rolling_upgrade": false
    }
  ]
}

If an index size is less than 10 GB and contains results from multiple jobs that are still required, we recommend reindexing into a new format using the UI. You can use the Get index information API to obtain the size of an index:

GET _cat/indices/.ml-anomalies-custom-example?v&h=index,store.size

Use the Upgrade Assistant to initiate reindexing.

If an index size is greater than 10 GB, we recommend using the Reindex API. Reindexing consists of the following steps:

1. Set the original index to read-only.

PUT .ml-anomalies-custom-example/_block/read_only

2. Create a new index from the legacy index.

POST _create_from/.ml-anomalies-custom-example/.reindexed-v9-ml-anomalies-custom-example

3. Reindex documents. To accelerate the reindexing process, we recommend setting the number of replicas to 0 before reindexing, then set it back to the original number once completed.

   1. Get the number of replicas.

   GET /.reindexed-v9-ml-anomalies-custom-example/_settings
   

   Note the number of replicas in the response. For example:

   {
     ".reindexed-v9-ml-anomalies-custom-example": {
       "settings": {
         "index": {
           "number_of_replicas": "1",
           "number_of_shards": "1"
         }
       }
     }
   }
   

   2. Set the number of replicas to 0. You must also set the auto_expand_replicas parameter to false to change the number of replicas:

   PUT /.reindexed-v9-ml-anomalies-custom-example/_settings
   {
     "index": {
       "auto_expand_replicas": false,
       "number_of_replicas": 0
     }
   }
   

   3. Start the reindexing process in asynchronous mode.

   POST _reindex?wait_for_completion=false
   {
     "source": {
       "index": ".ml-anomalies-custom-example"
     },
     "dest": {
       "index": ".reindexed-v9-ml-anomalies-custom-example"
     }
   }
   

   The response will contain a task_id. You can check when the task is completed using the following command:

   GET _tasks/<task_id>
   

   4. Set the number of replicas to the original number when the reindexing is finished. Optionally, you can set the auto_expand_replicas parameter back to its default value (0-1) to allow the number of replicas to be automatically adjusted based on the number of data nodes in the cluster.

   PUT /.reindexed-v9-ml-anomalies-custom-example/_settings
   {
     "index": {
       "number_of_replicas": "<original_number_of_replicas>",
       "auto_expand_replicas": "0-1"
     }
   }
   

4. Get the aliases the original index is pointing to.

GET .ml-anomalies-custom-example/_alias

The response may contain multiple aliases if the results of multiple jobs are stored in the same index.

{
  ".ml-anomalies-custom-example": {
    "aliases": {
      ".ml-anomalies-example1": {
        "filter": {
          "term": {
            "job_id": {
              "value": "example1"
            }
          }
        },
        "is_hidden": true
      },
      ".ml-anomalies-example2": {
        "filter": {
          "term": {
            "job_id": {
              "value": "example2"
            }
          }
        },
        "is_hidden": true
      }
    }
  }
}

5. Now you can reassign the aliases to the new index and delete the original index in one step. Note that when adding the new index to the aliases, you must use the same filter and is_hidden parameters as the original index.

POST _aliases
{
  "actions": [
    {
      "add": {
        "index": ".reindexed-v9-ml-anomalies-custom-example",
        "alias": ".ml-anomalies-example1",
        "filter": {
          "term": {
            "job_id": {
              "value": "example1"
            }
          }
        },
        "is_hidden": true
      }
    },
    {
      "add": {
        "index": ".reindexed-v9-ml-anomalies-custom-example",
        "alias": ".ml-anomalies-example2",
        "filter": {
          "term": {
            "job_id": {
              "value": "example2"
            }
          }
        },
        "is_hidden": true
      }
    },
    {
      "remove": {
        "index": ".ml-anomalies-custom-example",
        "aliases": ".ml-anomalies-*"
      }
    },
    {
      "remove_index": {
        "index": ".ml-anomalies-custom-example"
      }
    },
    {
      "add": {
        "index": ".reindexed-v9-ml-anomalies-custom-example",
        "alias": ".ml-anomalies-custom-example",
        "is_hidden": true
      }
    }
  ]
}

Legacy indices created in Elasticsearch 7.x can be made read-only and supported in Elasticsearch 9.x. Making an index with a large amount of historical results read-only allows for a quick migration to the next major release, since you don’t have to wait for the data to be reindexed into the new format. However, it has the limitation that even after deleting an anomaly detection job, the historical results associated with this job are not completely deleted. Therefore, the system will prevent you from creating a new job with the same name.

To set the index as read-only, add the write block to the index:

PUT .ml-anomalies-custom-example/_block/write

Indices created in Elasticsearch 7.x that have a write block will not raise a critical deprecation warning.

If an index contains results of the jobs that are no longer required. To list all jobs that stored results in an index, use the terms aggregation:

GET .ml-anomalies-custom-example/_search
{
  "size": 0,
  "aggs": {
    "job_ids": {
      "terms": {
        "field": "job_id",
        "size": 100
      }
    }
  }
}

The jobs can be deleted in the UI. After the last job is deleted, the index will be deleted as well.

To identify indices that require action, use the Deprecation info API:

GET /_migration/deprecations

The response contains the list of critical deprecation warnings in the index_settings section:

"index_settings": {
    "my-destination-index": [
      {
        "level": "critical",
        "message": "One or more Transforms write to this index with a compatibility version < 9.0",
        "url": "https://www.elastic.co/guide/en/elasticsearch/reference/master/migrating-9.0.html#breaking_90_transform_destination_index",
        "details": "Transforms [my-transform] write to this index with version [7.8.23].",
        "resolve_during_rolling_upgrade": false
      }
    ]
  }

If the index was created by the transform, you can use the Transform Reset API to delete the destination index and recreate it the next time the transform runs.

If the index was not created by the transform and you still want to reset it, you can manually delete and recreate the index, then call the Reset API.

POST _transform/my-transform/_reset

When the Upgrade Assistant reindexes the documents, Kibana will put a write block on the old destination index, copy the results to a new index, delete the old index, and create an alias to the new index. During this time, the transform will pause and wait for the destination to become writable again. If you do not want the transform to pause, continue to reindexing the transform’s destination index while the transform is running.

If an index size is less than 10 GB, we recommend using the Upgrade Assistant to automatically migrate the index. You can use the Get index information API to obtain the size of an index:

GET _cat/indices/.transform-destination-example?v&h=index,store.size

Use the Upgrade Assistant to initiate reindexing.

If an index size is greater than 10 GB, we recommend using the Reindex API. Reindexing consists of the following steps:

1. Set the original index to read-only.

PUT .transform-destination-example/_block/read_only

2. Create a new index from the legacy index.

POST _create_from/.transform-destination-example/.reindexed-v9-transform-destination-example

3. Reindex documents. To accelerate the reindexing process, we recommend setting the number of replicas to 0 before reindexing, then set it back to the original number once completed.

   1. Get the number of replicas.

   GET /.reindexed-v9-transform-destination-example/_settings
   

   Note the number of replicas in the response. For example:

   {
     ".reindexed-v9-transform-destination-example": {
       "settings": {
         "index": {
           "number_of_replicas": "1",
           "number_of_shards": "1"
         }
       }
     }
   }
   

   2. Set the number of replicas to 0.

   PUT /.reindexed-v9-transform-destination-example/_settings
   {
     "index": {
       "number_of_replicas": 0
     }
   }
   

   3. Start the reindexing process in asynchronous mode.

   POST _reindex?wait_for_completion=false
   {
     "source": {
       "index": ".transform-destination-example"
     },
     "dest": {
       "index": ".reindexed-v9-transform-destination-example"
     }
   }
   

   The response will contain a task_id. You can check when the task is completed using the following command:

   GET _tasks/<task_id>
   

   4. Set the number of replicas to the original number when the reindexing is finished.

   PUT /.reindexed-v9-transform-destination-example/_settings
   {
     "index": {
       "number_of_replicas": "<original_number_of_replicas>"
     }
   }
   

4. Get the aliases the original index is pointing to.

GET .transform-destination-example/_alias

The response may contain multiple aliases if the results of multiple jobs are stored in the same index.

{
  ".transform-destination-example": {
    "aliases": {
      ".transform-destination-example1": {
        "filter": {
          "term": {
            "job_id": {
              "value": "example1"
            }
          }
        },
        "is_hidden": true
      },
      ".transform-destination-example2": {
        "filter": {
          "term": {
            "job_id": {
              "value": "example2"
            }
          }
        },
        "is_hidden": true
      }
    }
  }
}

5. Now you can reassign the aliases to the new index and delete the original index in one step. Note that when adding the new index to the aliases, you must use the same filter and is_hidden parameters as the original index.

POST _aliases
{
  "actions": [
    {
      "add": {
        "index": ".reindexed-v9-transform-destination-example",
        "alias": ".transform-destination-example1",
        "filter": {
          "term": {
            "job_id": {
              "value": "example1"
            }
          }
        },
        "is_hidden": true
      }
    },
    {
      "add": {
        "index": ".reindexed-v9-transform-destination-example",
        "alias": ".transform-destination-example2",
        "filter": {
          "term": {
            "job_id": {
              "value": "example2"
            }
          }
        },
        "is_hidden": true
      }
    },
    {
      "remove": {
        "index": ".transform-destination-example",
        "aliases": ".transform-destination-*"
      }
    },
    {
      "remove_index": {
        "index": ".transform-destination-example"
      }
    },
    {
      "add": {
        "index": ".reindexed-v9-transform-destination-example",
        "alias": ".transform-destination-example",
        "is_hidden": true
      }
    }
  ]
}

If you want the transform and the reindex task to write documents to the new destination index at the same time:

1. Set the original index to read-only.

POST _create_from/my-destination-index/my-new-destination-index

2. Update the transform to write to the new destination index:

POST _transform/my-transform/_update
{
 "dest": {
   "index": "my-new-destination-index"
 }
}

3. Reindex documents. To accelerate the reindexing process, we recommend setting the number of replicas to 0 before reindexing, then set it back to the original number once completed.

   1. Get the number of replicas.

   GET /my-destination-index/_settings
   

   2. Note the number of replicas in the response. For example:

   {
     "my-destination-index":: {
       "settings": {
         "index": {
           "number_of_replicas": "1",
           "number_of_shards": "1"
         }
       }
     }
   }
   

   3. Set the number of replicas to 0.

   PUT /my-destination-index/_settings
   {
     "index": {
       "number_of_replicas": 0
     }
   }
   

   4. Start the reindexing process in asynchronous mode. Set the op_type to create so the reindex does not overwrite work that the transform is doing.

   POST _reindex
   {
     "conflicts": "proceed",
     "source": {
       "index": "my-destination-index"
     },
     "dest": {
       "index": "my-new-destination-index",
       "op_type": "create"
     }
   }
   

   The response will contain a task_id. You can check when the task is completed using the following command:

   GET _tasks/<task_id>
   

   5. Set the number of replicas to the original number when the reindexing is finished.

   PUT /my-new-destination-index/_settings
   {
     "index": {
       "number_of_replicas": "<original_number_of_replicas>"
     }
   }
   

4. Get the aliases the original index is pointing to.

   GET my-destination-index/_alias
   {
     "my-destination-index": {
       "aliases": {
         "my-destination-alias": {},
       }
     }
   }
   

5. Now you can reassign the aliases to the new index and delete the original index in one step. Note that when adding the new index to the aliases, you must use the same filter and is_hidden parameters as the original index.

   POST _aliases
   {
     "actions": [
       {
         "add": {
           "index": "my-new-destination-index",
           "alias": "my-destination-alias"
         }
       },
       {
         "remove": {
           "index": "my-destination-index",
           "aliases": "my-destination-alias"
         }
       },
       {
         "remove_index": {
           "index": "my-destination-index"
         }
       }
     ]
   }
   

You can use the Transform Delete API to delete the transform and stop it from writing to the destination index.

DELETE _transform/my-transform

If the destination index is no longer needed, it can be deleted with the transform.

DELETE _transform/my-transform?delete_dest_index