[Tracker] Improving cuvs-lucene release robustness

[imotov]

Java compatibility and support

• Raise minimum JDK for full functionality from 22 to 25 (LTS)
• Keep JDK 21 compile-time support and add a CI job that builds/runs on JDK 21
• Clearly document the JDK support policy (compile vs. run requirements)

Lucene support

• Support Lucene 10.2, 10.3, 10.4, 10.5 simultaneously - repurpose [FEA] Update cuvs-lucene to Lucene 10.5 #2410 and expand with:
  1. Restructure cuvs-lucene code to support different Lucene API for each supported version
  2. restructure cuvs-lucene build to publish one JAR per Lucene minor (cuvs-lucene-10.N-<ver>.jar)
  3. Compile against lowest patch version, test against highest patch version per minor
• Define a policy for adding new Lucene minors and dropping trailing ones

libcuvs ↔ cuvs-java ↔ cuvs-lucene compatibility

• [DOC] Add compatiblity matrix for ABI stability for versions #1682 (Document libcuvs ↔ cuvs-java ↔ cuvs-lucene compatibility rules)
• [FEA] [Java] Implement cuvs version check #1020
• Add a CI matrix testing supported libcuvs × cuvs-java × cuvs-lucene combinations

API compatibility

• Publish an API compatibility policy for cuvs-java/cuvs-lucene (no guarantee across minors, Lucene-style)

Index file format compatibility

• Publish an index format backward-compatibility guarantee (readable through the next major of lucene)
• Start working on a corpus of indices from older releases and test reading them or add a harness to be able to generate old indices on demand.
• Add backward-compatibility test matrix across Lucene versions × cuvs releases

Cross-platform compatibility

• Add explicit runtime checks + error messages for missing CUDA/insufficient VRAM when opening GPU indices
• Document platform/portability limits of GPU index files (snapshot/restore across clusters)

Performance guarantees

• [FEA] cuVS-Lucene Backend for cuVS-Bench #2194
• [FEA] Add cuvs-lucene and elastic backends to cuvs_bench #1856
• [FEA] Add lucene-HNSW and lucene-CAGRA-HNSW in cuVS Bench #1623
• [FEA] Elasticsearch backend for cuvs-bench #2195
• [FEA] cuvs-lucene backend for Apache Solr #2428
• [FEA] Set up regression benchmark #2014
• [FEA] Web service for nightly/weekly benchmarks #2180 Ideally also stand up a benchmark-results dashboard
• Expand benchmarks to run across different GPUs (RTX Pro 6000, H100, A10, L40/L40S) and multiple datasets

Testing

• Add multithreaded concurrency coverage for cuVS-Lucene PyLucene tests #2407 Expand with updates and deletes for some records
• Expand reproducible randomized testing with seeds in cuvs-lucene, surface test seeds in CI logs and treat seed-specific failures as real bugs, not flaky
• [FEA] Set up infra for scheduling e2e vector database benchmarks / integration tests #2105