Qualcomm: stop an expired bundle from making the backend cache stale (pytorch#21944)

QnnBackendUnifiedRegistry caches backend bundles as weak_ptr keyed on
backend type,
and CleanupExpired() only runs in the registry destructor. When a
bundle's last
owner goes away, the dead weak_ptr stays in the map. The next request
finds the
entry, fails to lock it, builds a fresh bundle, and then calls emplace()
— which is
a no-op because the key already exists. The corpse is never replaced, so
every
subsequent lowering rebuilds the backend and device instead of sharing
them.

insert_or_assign replaces the expired entry. CleanupExpired() can't be
used here
  because it takes mutex_, which is already held.

Single-lowering processes never reach this, which is why it has been
invisible. It
shows up once a process does two or more lowerings: with the fix the
second
lowering reuses the cached bundle instead of creating a second one,
which also
  avoids repeated device configuration.

Test: test_backend_bundle_cache_survives_an_expired_entry asserts two
creations
  plus at least one reuse, and fails on unmodified main.

cc @cbilgin

Author: psiddh

backends/qualcomm/runtime/backends/QnnBackendUnifiedRegistry.cpp

@@ -167,7 +167,11 @@ Error QnnBackendUnifiedRegistry::GetOrCreateBackendBundle(
   bundle->qnn_logger_ptr = std::move(logger);
   bundle->qnn_backend_ptr = std::move(backend);
   bundle->qnn_device_ptr = std::move(device);
-  qnn_backend_bundles_map_.emplace(
+  // insert_or_assign rather than emplace: an expired bundle leaves a dead
+  // weak_ptr under this key, and emplace will not replace it, so every later
+  // request would miss the cache and build another backend for the same type.
+  // CleanupExpired() cannot be used from here -- it takes mutex_, already held.
+  qnn_backend_bundles_map_.insert_or_assign(
       backend_type, bundle); // Store weak_ptr to the bundle
 
   return Error::Ok;

backends/qualcomm/tests/test_passes.py

@@ -795,6 +795,58 @@ def test_custom_op_resolves_supported_types(self):
                 _resolve_qnn_data_type(py_type, "arg", "my_ops.foo.default")
             )
 
+    def test_backend_bundle_cache_survives_an_expired_entry(self):
+        """A backend bundle that expires must not poison the cache.
+
+        QnnBackendUnifiedRegistry holds bundles by weak_ptr keyed on backend type.
+        The stale key left behind by an expired bundle was never replaced, so once
+        one bundle had come and gone every later request built a fresh backend --
+        even while another bundle for that type was alive.
+        """
+        import os
+        import tempfile
+
+        import executorch.backends.qualcomm.python.PyQnnManagerAdaptor as PyQnnManager
+        from executorch.backends.qualcomm.partition.utils import (
+            generate_qnn_executorch_option,
+        )
+
+        option = generate_qnn_executorch_option(
+            generate_qnn_executorch_compiler_spec(
+                soc_model=QcomChipset.SM8650,
+                backend_options=generate_htp_compiler_spec(use_fp16=True),
+            )
+        )
+
+        # QNN logs from C++, so capture at the fd level.
+        with tempfile.TemporaryFile() as cap:
+            saved = os.dup(2)
+            os.dup2(cap.fileno(), 2)
+            try:
+                first = PyQnnManager.QnnManager(option)
+                if first.InitBackend().value != 0:
+                    self.skipTest("QNN backend unavailable")
+                first.Destroy()
+                del first
+
+                kept = PyQnnManager.QnnManager(option)
+                self.addCleanup(kept.Destroy)
+                self.assertEqual(0, kept.InitBackend().value)
+
+                later = PyQnnManager.QnnManager(option)
+                self.addCleanup(later.Destroy)
+                self.assertEqual(0, later.InitBackend().value)
+            finally:
+                os.dup2(saved, 2)
+                os.close(saved)
+            cap.seek(0)
+            log = cap.read().decode("utf-8", "replace")
+
+        created = log.count("Creating new backend bundle")
+        reused = log.count("Use cached backend bundle")
+        self.assertEqual(2, created, f"expected 2 backend creations, got {created}")
+        self.assertGreaterEqual(reused, 1, "third manager must reuse the live bundle")
+
 
 if __name__ == "__main__":
     unittest.main()