fix #6150: A deadlock risk when space creation happens concurrently with leader balance in NebulaGraph

[cccxgit]

What type of PR is this?

• bug
• feature
• enhancement

What problem(s) does this PR solve?

Issue(s) number:

Issue(s) number: #6150

Description:

This PR fixes a deadlock risk when space creation happens concurrently with leader balance in NebulaGraph 3.6.

NebulaStore::addSpace() used to hold NebulaStore::lock_ as a write lock and then wait on newEngineAsync().get(). Since newEngineAsync() runs on folly::getGlobalIOExecutor(), and transfer-leader follow-up checks also run on the same executor and call partLeader() (which needs the store lock), this could create a lock/executor dependency cycle under concurrency.

How do you solve it?

• Extract engine creation logic into a synchronous helper createEngine()
• Keep newEngineAsync() for the startup path that scans existing engines from disk
• Change newEngine() to call createEngine() directly instead of waiting on newEngineAsync().get()

This removes the dependency on folly::getGlobalIOExecutor() from the addSpace() path while the store write lock is held.

Special notes for your reviewer, ex. impact of this fix, design document, etc:

The change only affects the synchronous space-creation path. Startup-time async engine loading remains unchanged.

Checklist:

Tests:

• Unit test(positive and negative cases)
• Function test
• Performance test
• N/A

Affects:

• Documentation affected (Please add the label if documentation needs to be modified.)
• Incompatibility (If it breaks the compatibility, please describe it and add the label.）
• If it's needed to cherry-pick (If cherry-pick to some branches is required, please label the destination version(s).)
• Performance impacted: Consumes more CPU/Memory

Release notes:

Please confirm whether to be reflected in release notes and how to describe:

fix bug: A deadlock risk when space creation happens concurrently with leader balance in NebulaGraph 3.6.

[CLAassistant]

All committers have signed the CLA.

[Copilot]

Pull request overview

This PR fixes a deadlock risk during concurrent space creation and leader-balance operations by removing addSpace()’s dependency on folly::getGlobalIOExecutor() while the store write lock is held.

Changes:

• Extract KV engine construction into a synchronous helper NebulaStore::createEngine().
• Update newEngineAsync() to delegate engine construction to createEngine() (preserving async startup behavior).
• Update newEngine() to construct engines synchronously via createEngine() instead of blocking on newEngineAsync().get().

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
src/kvstore/NebulaStore.h	Declares the new synchronous createEngine() helper used to decouple space creation from the global IO executor.
src/kvstore/NebulaStore.cpp	Implements createEngine() and refactors newEngine()/newEngineAsync() to use it, eliminating the lock/executor dependency cycle in the addSpace() path.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[cccxgit]

@xiajingchun 你好请问 ，流水线执行失败，是因为流水线本身问题还是测试用例没通过呢？

[xiajingchun]

@xiajingchun 你好请问 ，流水线执行失败，是因为流水线本身问题还是测试用例没通过呢？

@cccxgit 之前workflow有些问题，现在好了。但是lint的format check没过，你再改改：

Run git diff -U0 --no-color c1d657d3811892b716e2bbb76f86690186129335 HEAD | /usr/share/clang/clang-format-10/clang-format-diff.py -p1 | tee /tmp/.clang-format-diff
  
--- src/kvstore/NebulaStore.cpp	(before formatting)
+++ src/kvstore/NebulaStore.cpp	(after formatting)
@@ -368,12 +368,8 @@
       cfFactory = options_.cffBuilder_->buildCfFactory(spaceId);
     }
     auto vIdLen = getSpaceVidLen(spaceId);
-    engine = std::make_unique<RocksEngine>(spaceId,
-                                           vIdLen,
-                                           dataPath,
-                                           walPath,
-                                           options_.mergeOp_,
-                                           cfFactory);
+    engine = std::make_unique<RocksEngine>(
+        spaceId, vIdLen, dataPath, walPath, options_.mergeOp_, cfFactory);
   } else {
     LOG(FATAL) << "Unknown engine type " << FLAGS_engine_type;
   }
Error: Process completed with exit code 1.

[xiajingchun]

@cccxgit still failing:

Run git diff -U0 --no-color c1d657d3811892b716e2bbb76f86690186129335 HEAD | /usr/share/clang/clang-format-10/clang-format-diff.py -p1 | tee /tmp/.clang-format-diff
--- src/kvstore/NebulaStore.cpp	(before formatting)
+++ src/kvstore/NebulaStore.cpp	(after formatting)
@@ -369,7 +369,7 @@
     }
     auto vIdLen = getSpaceVidLen(spaceId);
     engine = std::make_unique<RocksEngine>(
-      spaceId, vIdLen, dataPath, walPath, options_.mergeOp_, cfFactory);
+        spaceId, vIdLen, dataPath, walPath, options_.mergeOp_, cfFactory);
   } else {
     LOG(FATAL) << "Unknown engine type " << FLAGS_engine_type;
   }
Error: Process completed with exit code 1.

[cccxgit]

@cccxgit仍然失败：

Run git diff -U0 --no-color c1d657d3811892b716e2bbb76f86690186129335 HEAD | /usr/share/clang/clang-format-10/clang-format-diff.py -p1 | tee /tmp/.clang-format-diff
--- src/kvstore/NebulaStore.cpp	(before formatting)
+++ src/kvstore/NebulaStore.cpp	(after formatting)
@@ -369,7 +369,7 @@
     }
     auto vIdLen = getSpaceVidLen(spaceId);
     engine = std::make_unique<RocksEngine>(
-      spaceId, vIdLen, dataPath, walPath, options_.mergeOp_, cfFactory);
+        spaceId, vIdLen, dataPath, walPath, options_.mergeOp_, cfFactory);
   } else {
     LOG(FATAL) << "Unknown engine type " << FLAGS_engine_type;
   }
Error: Process completed with exit code 1.

已按照建议修改，感谢哈

[cccxgit]

@xiajingchun 不好意思打扰，流水线好像还有问题