Skip to content

Conversation

@punAhuja
Copy link

https://issues.apache.org/jira/browse/SOLR-17990

This PR integrates FAISS (Facebook AI Similarity Search) into Apache Solr for vector search. This implementation backports FaissKnnVectorsFormat from Lucene 11.0 (unreleased main branch) to work with Lucene 10.3.1 used by Solr 9.2.0 and branch_10_0.

Implementation:

  • Backported FAISS codec implementation from Lucene 11.0 (org.apache.lucene.sandbox.codecs.faiss)
  • Added Solr integration layer (org.apache.solr.faiss.FaissCodecFactory, FaissCodec)
  • Implemented Java Foreign Function & Memory (FFM) API integration for native FAISS library (libfaiss_c.so version 1.11.0)
  • Added FFMUtils compatibility layer for Java 21/22+ FFM API differences

Version Compatibility

  • Production code: Uses Lucene 10.3.1 (matches Solr 9.2.0)
  • Issue: Published Solr 9.2.0 test framework uses Lucene 9.4.2, causing binary incompatibility
  • Solution: Added Git submodule pointing to Solr branch_10_0 (uses Lucene 10.3.1) for test framework compatibility
  • Submodule builds required JARs locally; build system detects and uses them automatically

Testing

  • Added integration test TestFaissIntegration verifying codec loading, indexing, and search functionality
  • Tests require FAISS native library (libfaiss_c.so version 1.11.0) in library path
  • Tests automatically use submodule JARs when available, fall back to published artifacts with warnings

- Backport FaissKnnVectorsFormat from Lucene 11.0 to Lucene 10.3.1
- Add FaissCodecFactory and FaissCodec for Solr integration
- Add Solr branch_10_0 submodule for test compatibility
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant