Do not fail on duplicate WhatsApp IDs in the iOS address book - #236
Open
mfcarroll wants to merge 1 commit into
Open
Do not fail on duplicate WhatsApp IDs in the iOS address book#236mfcarroll wants to merge 1 commit into
mfcarroll wants to merge 1 commit into
Conversation
ios_handler.contacts() constructs a ChatStore for every row of
ZWAADDRESSBOOKCONTACT and passes each one to ChatCollection.add_chat(), which
raises ValueError when the chat ID is already present. The iOS address book can
map several rows to one WhatsApp ID, most commonly the same number saved under
more than one name, so the export aborts before any chat is processed:
ValueError: Chat ID already exists. Use get_chat to retrieve existing chat.
The database this was found on has 125 such WhatsApp IDs out of 3,476 rows, one
of them appearing eight times, which made the export impossible to run at all.
Rows for an ID that is already present now enrich the existing chat rather than
adding a second one: a name or status is filled in only where the existing chat
has none, and a ZLID that has not been seen yet is registered as an alias. First
non-empty value wins, which keeps the outcome stable regardless of row order.
That ordering matters because duplicates frequently disagree: 121 of the 125 in
that database have differing ZFULLNAME values. Names are in any case refined
afterwards by messages(), which prefers a real name over a phone number.
add_chat() keeps raising on a duplicate ID so that it still catches genuine
programming errors elsewhere.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pre-flight Check
All pull requests (excluding those with changes unrelated to source files) must target and branch off from the
devbranch.Please select the applicable option below:
devbranch.Related Issue
Description of Changes
ios_handler.contacts()builds aChatStorefor every row ofZWAADDRESSBOOKCONTACTand passes each one toChatCollection.add_chat(), which raises when the chat ID is already present:The iOS address book can map several rows to a single WhatsApp ID — most commonly the same number saved under more than one name. When that happens the export aborts during contact pre-processing, before a single chat is written. On the database I hit this with, 125 WhatsApp IDs were each shared by more than one row — 282 of 3,476 rows in total, one ID appearing eight times — which made
devimpossible to run at all for that backup.Reproducer, if useful:
The change
A row whose WhatsApp ID is already present now enriches the existing chat instead of adding a second one for the same ID:
nameandstatusare filled in only where the existing chat has none, so the first non-empty value wins.ZLIDnot yet seen for that chat is registered as an alias, so a LID appearing only on a later duplicate row is not lost.add_chat()is left as it is, still raising on a duplicate ID, so it continues to catch genuine programming errors at the other call sites.I deliberately kept this to "first non-empty wins" rather than trying to pick the best row. Duplicates frequently disagree — 121 of the 125 in my database had differing
ZFULLNAMEvalues, and there is no signal in the table that says which is authoritative. Deterministic behaviour that does not depend on row order seemed better than a guess, and names are refined afterwards bymessages(), which already prefers a real name over a phone number.Testing
tests/test_ios_handler.pywith 8 tests: duplicates do not raise, first non-empty values are kept, a LID from a later duplicate becomes an alias and resolves to the same chat, a repeated LID is not added twice, distinct IDs stay separate, bare numbers are normalised before deduplication, null IDs are skipped, and the code still works on older schemas with noZLIDcolumn. 6 of the 8 fail without the fix.devin the same environment (test_sanity_checkneeds a compiled binary; thedetermine_dayandTimingfailures are timezone-dependent, the latter being what fix: crash in timestamp formatting when timezone_offset is none #205 / fix: handle None timezone_offset in timestamp formatting #208 address).Pre-processed 3476 contacts, export completes.Notes
Two things I noticed but did not touch, to keep this focused:
add_chat()call sites —android_handler.contacts()(line 54) andvcards_contacts.py(line 37), where a.vcfcontaining two entries for one number would crash the same way. I have no Android or multi-entry vCard sample to verify a change against, so I left them alone rather than guess. Happy to follow up if you want them covered, or if you would prefer the deduplication to live insideadd_chat()instead of at the call sites — that is a design call I did not want to make unilaterally, since it would remove a guard the other call sites rely on.ios_handler.pyline 348 emitsSyntaxWarning: "is not" with 'str' literal(if entity is not "Someone"). Unrelated to this change, but it is a latent bug: identity comparison against a string literal is not guaranteed to be true even when the values are equal.There may be a small overlap with #233, which also adds
tests/test_ios_handler.py. Both only append test functions, so whichever lands second should be a trivial merge.