Skip to content

Do not fail on duplicate WhatsApp IDs in the iOS address book - #236

Open
mfcarroll wants to merge 1 commit into
KnugiHK:devfrom
mfcarroll:fix/ios-duplicate-contact-ids
Open

Do not fail on duplicate WhatsApp IDs in the iOS address book#236
mfcarroll wants to merge 1 commit into
KnugiHK:devfrom
mfcarroll:fix/ios-duplicate-contact-ids

Conversation

@mfcarroll

@mfcarroll mfcarroll commented Aug 6, 2026

Copy link
Copy Markdown

Pre-flight Check

All pull requests (excluding those with changes unrelated to source files) must target and branch off from the dev branch.

Please select the applicable option below:

  • This PR does not modify any source files (e.g., only updates the README).
  • This PR modifies one or more source files and targets the dev branch.

Related Issue

  • No existing issue — I could not find one for this crash. Happy to open one if you would rather track it separately.

Description of Changes

ios_handler.contacts() builds a ChatStore for every row of ZWAADDRESSBOOKCONTACT and passes each one to ChatCollection.add_chat(), which raises when the chat ID is already present:

ValueError: Chat ID already exists. Use get_chat to retrieve existing chat.

The iOS address book can map several rows to a single WhatsApp ID — most commonly the same number saved under more than one name. When that happens the export aborts during contact pre-processing, before a single chat is written. On the database I hit this with, 125 WhatsApp IDs were each shared by more than one row — 282 of 3,476 rows in total, one ID appearing eight times — which made dev impossible to run at all for that backup.

Reproducer, if useful:

CREATE TABLE ZWAADDRESSBOOKCONTACT (ZWHATSAPPID TEXT, ZLID TEXT, ZFULLNAME TEXT, ZABOUTTEXT TEXT);
INSERT INTO ZWAADDRESSBOOKCONTACT VALUES ('1555000@s.whatsapp.net', NULL, 'First Name',  NULL);
INSERT INTO ZWAADDRESSBOOKCONTACT VALUES ('1555000@s.whatsapp.net', NULL, 'Second Name', NULL);

The change

A row whose WhatsApp ID is already present now enriches the existing chat instead of adding a second one for the same ID:

  • name and status are filled in only where the existing chat has none, so the first non-empty value wins.
  • A ZLID not yet seen for that chat is registered as an alias, so a LID appearing only on a later duplicate row is not lost.
  • add_chat() is left as it is, still raising on a duplicate ID, so it continues to catch genuine programming errors at the other call sites.

I deliberately kept this to "first non-empty wins" rather than trying to pick the best row. Duplicates frequently disagree — 121 of the 125 in my database had differing ZFULLNAME values, and there is no signal in the table that says which is authoritative. Deterministic behaviour that does not depend on row order seemed better than a guess, and names are refined afterwards by messages(), which already prefers a real name over a phone number.

Testing

  • New tests/test_ios_handler.py with 8 tests: duplicates do not raise, first non-empty values are kept, a LID from a later duplicate becomes an alias and resolves to the same chat, a repeated LID is not added twice, distinct IDs stay separate, bare numbers are normalised before deduplication, null IDs are skipped, and the code still works on older schemas with no ZLID column. 6 of the 8 fail without the fix.
  • Full suite: 92 passed. The 5 failures are identical to the 5 that fail on unmodified dev in the same environment (test_sanity_check needs a compiled binary; the determine_day and Timing failures are timezone-dependent, the latter being what fix: crash in timestamp formatting when timezone_offset is none #205 / fix: handle None timezone_offset in timestamp formatting #208 address).
  • Verified against the real 3,476-contact database that triggered the crash: Pre-processed 3476 contacts, export completes.

Notes

Two things I noticed but did not touch, to keep this focused:

  1. The same hazard exists at two other unguarded add_chat() call sites — android_handler.contacts() (line 54) and vcards_contacts.py (line 37), where a .vcf containing two entries for one number would crash the same way. I have no Android or multi-entry vCard sample to verify a change against, so I left them alone rather than guess. Happy to follow up if you want them covered, or if you would prefer the deduplication to live inside add_chat() instead of at the call sites — that is a design call I did not want to make unilaterally, since it would remove a guard the other call sites rely on.
  2. ios_handler.py line 348 emits SyntaxWarning: "is not" with 'str' literal (if entity is not "Someone"). Unrelated to this change, but it is a latent bug: identity comparison against a string literal is not guaranteed to be true even when the values are equal.

There may be a small overlap with #233, which also adds tests/test_ios_handler.py. Both only append test functions, so whichever lands second should be a trivial merge.

ios_handler.contacts() constructs a ChatStore for every row of
ZWAADDRESSBOOKCONTACT and passes each one to ChatCollection.add_chat(), which
raises ValueError when the chat ID is already present. The iOS address book can
map several rows to one WhatsApp ID, most commonly the same number saved under
more than one name, so the export aborts before any chat is processed:

    ValueError: Chat ID already exists. Use get_chat to retrieve existing chat.

The database this was found on has 125 such WhatsApp IDs out of 3,476 rows, one
of them appearing eight times, which made the export impossible to run at all.

Rows for an ID that is already present now enrich the existing chat rather than
adding a second one: a name or status is filled in only where the existing chat
has none, and a ZLID that has not been seen yet is registered as an alias. First
non-empty value wins, which keeps the outcome stable regardless of row order.
That ordering matters because duplicates frequently disagree: 121 of the 125 in
that database have differing ZFULLNAME values. Names are in any case refined
afterwards by messages(), which prefers a real name over a phone number.

add_chat() keeps raising on a duplicate ID so that it still catches genuine
programming errors elsewhere.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant