Support for 7.x segments as archive in Elasticsearch 8.x #117042
Description
Description
Elasticsearch 8.x supports restoring unsupported versions as archive in read-only mode. That works well for indices created in 5.x or 6.x.
We recently discovered an issue around indices created with 6.x, that get written into by Elasticsearch 7.x. Such indices can't be read by Elasticsearch 8.x because they have been created in 6.x, yet when imported as archive they cause issues because some of their segments have been written into using Lucene 8.x codecs. The appropriate codec for reading these segments is available in the classpath and can be loaded via SPI from the ordinary Lucene backward codecs, as indices created with 7.x (Lucene 8.x codec) can be read in Elasticsearch 8.x, but then there is a mismatch in that their version is higher than the actual version that the index was created with according to the archive indices functionality. This scenario should be supported by the archive indices functionality and is currently not tested.
The error is the following:
Caused by: org.apache.lucene.index.CorruptIndexException: segments file recorded minSegmentLuceneVersion=9.8.0 but segment=_89tq1(8.11.1):C21884823:[diagnostics={java.vendor=Oracle Corporation, source=merge, os.arch=amd64, mergeFactor=10, java.runtime.version=18.0.1.1+2-6, os=Linux, timestamp=1720555549541, lucene.version=8.11.1, java.vm.version=18.0.1.1+2-6, java.version=18.0.1.1, mergeMaxNumSegments=-1, os.version=3.10.0-1160.88.1.el7.x86_64}]:[attributes={Lucene87StoredFieldsFormat.mode=BEST_SPEED}] has older version=8.11.1 (resource=BufferedChecksumIndexInput(NIOFSIndexInput(path="/app/data/indices/8mNgc2SwTLahgktcEMxW1Q/5/index/segments_27sz")))
at org.apache.lucene.index.SegmentInfos.parseSegmentInfos(SegmentInfos.java:466) ~[lucene-core-9.8.0.jar:?]
at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:364) ~[lucene-core-9.8.0.jar:?]
at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:300) ~[lucene-core-9.8.0.jar:?]
at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:289) ~[lucene-core-9.8.0.jar:?]
at org.elasticsearch.common.lucene.Lucene.readSegmentInfos(Lucene.java:152) ~[elasticsearch-8.11.4.jar:?]
at org.elasticsearch.common.lucene.Lucene.pruneUnreferencedFiles(Lucene.java:165) ~[elasticsearch-8.11.4.jar:?]
at org.elasticsearch.xpack.lucene.bwc.OldLuceneVersions.convertToNewFormat(OldLuceneVersions.java:195) ~[?:?]