[trueoneplusone]

Type

• Bug fix
• New feature (adds an optional low-memory execution mode)
• Breaking change

Related to #1600.

Motivation and Context

ClusterDBSCAN retains every radius neighborhood before clustering. Dense clouds can therefore require quadratic memory, as reported in #1600. The expansion work-list can also retain repeated copies of pending points; removing the neighborhood cache alone does not bound memory.

This adds precompute_neighbors=False to legacy and tensor point-cloud DBSCAN in C++ and Python. It performs exact radius queries on demand with O(N) auxiliary storage. The existing parallel precomputation remains the default. Dense clouds still require substantial computation, and the sequential search mode trades parallelism for lower memory use.

Checklist

• I have run python util/check_style.py --apply --changed-only --no_parallel to apply Open3D code style to my code.
• This PR changes Open3D behavior or adds new functionality.
  • Both C++ and Python documentation is updated accordingly.
  • I have added C++ and Python unit tests.
• I will follow up and update the code if CI fails.
• For fork PRs, I have selected Allow edits from maintainers.

Description

• Share one expansion implementation between cached and on-demand radius queries.
• Label points when discovered so each undefined point enters the work-list at most once. Previously visited noise can become a border point without another query.
• Preserve radius-search ordering, boundary semantics, deterministic cluster IDs, and the default call behavior.
• Forward the option through the existing tensor CPU wrapper and document the memory/time tradeoff in both tutorials.
• Add regression cases for dense neighborhoods, shared border points, noise promotion, duplicates, empty clouds, threshold boundaries, Float32/Float64, and unchanged input/device behavior.

Plan followed: lock exact-label compatibility and linear auxiliary storage; add API and semantic regressions; implement the shared query path and bounded frontier; validate both public APIs against edge cases and the real point-cloud fixture; benchmark pristine, default, and low-memory modes with identical build settings. The original three-argument C++ symbols are preserved as forwarding overloads; existing source and already-linked binary callers keep the default precomputed path. Python stays bound explicitly to the four-argument overload.

Validation

Built pristine b6c5e196 and this patch as Release shared libraries, C++ tests, and Python bindings with identical CPU-only configuration (Ubuntu 24.04 under WSL2, GCC 13.3, Python 3.12).

• ABI compatibility follow-up (9643cb4): exact three- and four-argument member-function signatures compile for both legacy and tensor point clouds. The rebuilt shared library exports all four symbols. The pristine pre-PR test binary was then run unchanged against the rebuilt library and both of its DBSCAN tests passed, proving existing linked callers still resolve the old symbols.

• ABI-fix validation: 5 focused C++ overload/edge-case tests passed; the Python DBSCAN suite passed 40 tests; changed-file style and whitespace checks passed.

• build/bin/tests --gtest_filter='*ClusterDBSCAN*': 5 passed, including the 196,133-point PLYPointCloud fixture and exact equality of both modes.

• pytest python/test/t/geometry/test_pointcloud.py -k cluster_dbscan: 40 passed. These exercise both public bindings, Float32/Float64 and CPU output-device preservation.

• Independent core-component/union-find oracle: 317 cases passed on pristine Open3D, and 634 comparisons passed across both patched modes, including seeded random clouds. This oracle does not reuse the expansion algorithm.

• Progress enabled/disabled: both modes and both APIs return identical labels in 8 additional smoke checks.

• Official style checks passed on Linux and Windows; both notebook edits are confined to Markdown documentation cells.

CUDA/SYCL and other operating-system builds were not run locally; this does not claim the complete Open3D test suite passed.

Measurements

Measurements use actual native C++ binaries, separately preserved pristine/patched libraries, GCC 13.3 Release, and the same build options. Hardware: AMD EPYC 9654, Ubuntu 24.04 under WSL2. Open3D/TBB and BLAS are limited to 8 threads. Compilation had finished before measuring; no CPU affinity was set.

Each cell is total process peak RSS (MiB) / clustering time (seconds), reporting medians of three fresh-process runs after one warmup, with rotated mode order. RSS comes from /usr/bin/time -v; clustering time excludes input generation/loading and label output. These native measurements do not include Python import memory.

Input	Pristine	Patched default	Patched low-memory
Dense synthetic, 3,000 points	78.68 / 0.1782	52.72 / 0.1383	24.30 / 0.6088
Dense synthetic, 6,000 points	250.17 / 0.6565	156.23 / 0.4556	24.27 / 3.1451
Dense synthetic, 12,000 points	944.90 / 2.8616	571.74 / 1.8816	24.27 / 21.8772
Sparse synthetic, 6,000 points	24.33 / 0.0046	24.27 / 0.0033	24.30 / 0.0021
60 synthetic groups, 6,000 points	24.26 / 0.0099	24.31 / 0.0091	24.31 / 0.0251
Real fragment, 196,133 points, eps=0.02	81.20 / 0.3304	70.70 / 0.2815	45.28 / 0.7903
Real fragment, 196,133 points, eps=0.05	313.84 / 1.3349	187.78 / 0.9293	45.26 / 3.5292

For the real cloud at eps=0.05, the opt-in mode reduces total peak RSS by 85.6%, with 2.64x the pristine clustering time. For the complete-neighborhood 12,000-point synthetic case, RSS falls 97.4%, with 7.65x the time. Its three streaming runs ranged from 21.38 to 24.87 seconds. The default mode also benefits from removing duplicate pending entries. Small control-case RSS differences are measurement noise, not claimed improvements.

All 84 processes completed under an 8-GiB address-space limit and a 180-second timeout. Full int32 label SHA256 values matched across all three implementations and every repetition for each input, including numeric cluster IDs.

The real input is the unchanged, complete official PLYPointCloud fragment.ply, MD5 831ecffd4d7cbbbe02494c5c351aa6e5, with min_points=10. Dense synthetic inputs have diameter <0.14 and eps=1, so every point is a neighbor of every other point. Sparse inputs are all noise; the groups control has 60 disconnected clusters. These results are a memory/time tradeoff, not a claim of universal speedup.

Compact C++ memory reproduction

After building/installing the patched library, save this as reproduce_dbscan_memory.cpp:

#include <chrono>
#include <iostream>
#include <string>
#include "open3d/geometry/PointCloud.h"
#include "open3d/utility/Parallel.h"

int main(int argc, char** argv) {
    const bool precompute = argc > 1 && std::string(argv[1]) == "precompute";
    const int n = argc > 2 ? std::stoi(argv[2]) : 6000;
    if (n < 10 || n > 12000) return 2;
    open3d::utility::SetMaxThreads(8);
    open3d::geometry::PointCloud cloud;
    for (int i = 0; i < n; ++i) {
        cloud.points_.emplace_back((i % 97) * 0.001,
                                   ((i / 97) % 97) * 0.001,
                                   (i / (97 * 97)) * 0.001);
    }
    const auto start = std::chrono::steady_clock::now();
    const auto labels = cloud.ClusterDBSCAN(1.0, 10, false, precompute);
    const auto elapsed = std::chrono::duration<double>(
            std::chrono::steady_clock::now() - start).count();
    if (labels.size() != static_cast<size_t>(n)) return 1;
    for (int label : labels) {
        if (label != 0) return 1;
    }
    std::cout << labels.size() << " points, one exact cluster, "
              << elapsed << " seconds\n";
}

Use this CMakeLists.txt:

cmake_minimum_required(VERSION 3.20)
project(dbscan_memory LANGUAGES CXX)
find_package(Open3D REQUIRED)
add_executable(dbscan_memory reproduce_dbscan_memory.cpp)
target_compile_features(dbscan_memory PRIVATE cxx_std_17)
target_link_libraries(dbscan_memory PRIVATE Open3D::Open3D)

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=/path/to/Open3D/install
cmake --build build -j8
export OPENBLAS_NUM_THREADS=8 OMP_NUM_THREADS=8
/usr/bin/time -v build/dbscan_memory precompute 6000
/usr/bin/time -v build/dbscan_memory streaming 6000

Repeat at 3,000 and 12,000 points. This short reproduction compares both modes of the patched library. To measure pristine Open3D, build the same program against a separately preserved pristine installation and replace the four-argument call with cloud.ClusterDBSCAN(1.0, 10, false). Each invocation checks the complete expected label vector. The table above uses a separate harness that also records all label bytes, library paths and repeated-run statistics.

[update-docs]

Thanks for submitting this pull request! The maintainers of this repository would appreciate if you could update the CHANGELOG.md based on your changes.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[kvnloo]

The low-memory path and validation look very thorough. One API/ABI detail seems worth addressing before merge.

Adding the defaulted "precompute_neighbors" parameter changes the C++ "ClusterDBSCAN" symbol, so this is source-compatible for rebuilt callers but not binary-compatible with code linked against the existing 3-argument overload. The PR description already notes that binary consumers would need to rebuild.

Since this is an additive opt-in mode and the existing 3-argument behavior remains exactly the default, could we preserve the current overload and add the new form separately?

For example, conceptually:

ClusterDBSCAN(double eps, size_t min_points,
bool print_progress = false) const;

ClusterDBSCAN(double eps, size_t min_points,
bool print_progress,
bool precompute_neighbors) const;

with the existing overload forwarding to the new implementation using "precompute_neighbors=true".

That would preserve the old exported symbol while still exposing the fourth argument through C++ and Python. It also seems aligned with the repository guidance to preserve existing public APIs unless the task requires changing them.

AI assistance disclosure: reviewed with OpenAI ChatGPT GPT-5.6 Sol.

[trueoneplusone]

Thanks for catching the ABI issue. The latest commit 9643cb4 retains the original three-argument ClusterDBSCAN overload in both legacy and tensor APIs, forwarding to the new four-argument overload with precompute_neighbors=true; Python binds the four-argument version explicitly. The rebuilt library exports both signatures, and the pristine pre-PR DBSCAN test binary ran unchanged against it (2/2 tests passed). Focused C++ (5) and Python (40) DBSCAN tests and changed-file style checks also passed. CHANGELOG.md and the PR description are updated with the ABI validation details.