[eric7h]

Related Issues

• No existing issue; found while reading FAISSDocumentStore.write_documents.

Proposed Changes:

When one write_documents call contains the same document ID twice (e.g. with DuplicatePolicy.OVERWRITE), the FAISS index ends up with an orphaned vector.

Root cause: vectors are collected in ids_to_add_to_index / vectors_to_add and only added to the index at the end of the call. When the second copy of an ID is processed, the overwrite path calls delete_documents([doc.id]), which drops the ID mapping and calls remove_ids — but the first copy's vector is not in the index yet, so nothing is removed. Its vector is then still added at the end under an int ID that no longer maps to any document.

Effects: index.ntotal is larger than count_documents(), and search() can return fewer than top_k results, because the orphan takes a slot in FAISS's top-k and is then skipped.

store = FAISSDocumentStore(embedding_dim=2)
store.write_documents([
    Document(id="a", content="old", embedding=[0.0, 0.0]),
    Document(id="a", content="new", embedding=[0.1, 0.1]),
    Document(id="b", content="b", embedding=[5.0, 5.0]),
], policy=DuplicatePolicy.OVERWRITE)
store.count_documents(), store.index.ntotal       # (2, 3)
[d.id for d in store.search([0.0, 0.0], top_k=2)]  # ['a']  -> expected ['a', 'b']

Fix: keep the pending vectors in a dict keyed by document ID and drop the pending entry when that ID is overwritten later in the same batch.

How did you test it?

• Added test_write_documents_overwrite_duplicate_in_same_batch_leaves_no_orphan_vector; it fails on main (ntotal == 3, search returns only a) and passes with the fix.
• pytest tests in integrations/faiss: 103 passed, 2 skipped.
• ruff check, ruff format --check and mypy (hatch types script) pass.

Notes for the reviewer

Small, self-contained change in write_documents; behaviour for batches without duplicate IDs is unchanged.

This PR was generated with an AI assistant. I have reviewed the changes and run the relevant tests.

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.

[CLAassistant]

All committers have signed the CLA.

[github-actions]

Coverage report (faiss)

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
integrations/faiss/src/haystack_integrations/document_stores/faiss
document_store.py
Project Total

This report was generated by python-coverage-comment-action

[HaystackBot]

Hi @eric7h, thanks a lot for your contribution! 🙏

We noticed that the Contributor License Agreement (CLA) check (license/cla) hasn't passed yet, so we've temporarily moved this PR to draft and paused the review assignment.

To get your PR reviewed, please sign the CLA via the link in the license/cla check below (or in the CLA bot comment). As soon as the check turns green, this PR will automatically be marked ready for review again and a reviewer will be re-assigned.

[HaystackBot]

Thanks for signing the CLA, @eric7h! 🎉 This PR is now ready for review again and the reviewer has been re-assigned.