Skip to content
This repository was archived by the owner on Jun 18, 2026. It is now read-only.

perf(matchImei): replace O(N*M) nested ResultSet scan with HashMap lookup - #99

Closed
sauravbhattacharya001 wants to merge 1 commit into
masterfrom
perf/hashmap-device-lookup
Closed

sauravbhattacharya001 wants to merge 1 commit into
masterfrom
perf/hashmap-device-lookup

Conversation

@sauravbhattacharya001

Copy link
Copy Markdown
Owner

Summary

Replace the nested-loop device matching in \matchImei.java\ with an in-memory HashMap for O(1) lookups.

Before

  • For each event row, the code rewound
    s_device\ to the first row and scanned linearly to find a match
  • Complexity: O(events × devices) per pass (sndrnode + srcnode)
  • Each matched row triggered an individual \�xecuteUpdate()\

After

  • Device table loaded once into a \HashMap<String, String>\ — O(devices)
  • Each event row does a single \HashMap.get()\ — O(1)
  • Overall: O(events + devices) per pass
  • Batch updates (flush every 500 rows) reduce DB round-trips
  • Transaction wrapping with rollback on failure
  • Extracted \matchAndUpdate()\ to eliminate duplicated sndrnode/srcnode logic
  • Removed \SCROLL_INSENSITIVE\ cursors (no longer needed without
    s.first()\ rewind)

…okup

Replace the nested-loop device matching (O(events × devices) per pass)
with an in-memory HashMap for O(1) lookups per event row, reducing
overall complexity to O(events + devices).

Additional improvements:
- Batch UPDATE statements (flush every 500) for fewer DB round-trips
- Wrap both passes in a transaction with rollback on failure
- Extract reusable matchAndUpdate() method to eliminate code duplication
- Remove SCROLL_INSENSITIVE cursors (no longer needed without rs.first() rewind)
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@github-actions github-actions Bot added data-pipeline Meeting extraction and processing pipeline size/m labels Mar 21, 2026
@sauravbhattacharya001

Copy link
Copy Markdown
Owner Author

Closing: superseded or conflicting with newer changes already on main/master.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

data-pipeline Meeting extraction and processing pipeline size/m

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant