Skip to content

Detach conditions from WaitSets before the vtable is rewritten - #6510

Open
PavelGuzenfeld wants to merge 1 commit into
eProsima:masterfrom
PavelGuzenfeld:fix/condition-vptr-race-3883
Open

PavelGuzenfeld wants to merge 1 commit into
eProsima:masterfrom
PavelGuzenfeld:fix/condition-vptr-race-3883

Conversation

@PavelGuzenfeld

Copy link
Copy Markdown

Description

Fixes #3883.

Condition::~Condition() calls ConditionNotifier::will_be_deleted(), which is
where a condition detaches itself from any WaitSet it is still attached to. That
runs after the derived destructor, so the vtable has already been rewritten by
the time the detach happens. Until will_be_deleted() manages to take the
WaitSet mutex, the condition is still in entries_, and a WaitSetImpl::wait()
holding that mutex will call get_trigger_value() on a partially destroyed
object.

TSan on a fully instrumented build, on master (2bad9bc):

WARNING: ThreadSanitizer: data race
  Read of size 8 at 0x720800004020 by thread T1 (mutexes: write M0):
    #0 operator() src/cpp/fastdds/core/condition/WaitSetImpl.cpp:124
    #3 WaitSetImpl::wait(...) src/cpp/fastdds/core/condition/WaitSetImpl.cpp:143
  Previous write of size 8 at 0x720800004020 by thread T2:
    #0 Condition::~Condition() src/cpp/fastdds/core/condition/Condition.cpp:32
    #1 GuardCondition::~GuardCondition() src/cpp/fastdds/core/condition/GuardCondition.cpp:34
  Location is heap block of size 24 at 0x720800004020 allocated by thread T2

WaitSetImpl.cpp:124 is c->get_trigger_value() inside fill_active_conditions.
Condition.cpp:32 is the vptr store on entry to ~Condition.

The change detaches in each derived destructor instead, while the object is still
fully constructed. ResourceLimitedVector::remove() returns false for a missing
element, so the later call from Condition::~Condition() is a no-op, and
notifier_ is a Condition member so it is still alive in a derived destructor
body. Applied to GuardCondition, StatusCondition and ReadCondition — every
Condition subclass in the tree. No ABI or API change.

How this was verified

Standalone reproducer below, Fast-DDS + Fast-CDR + foonathan_memory all built
with -fsanitize=thread, Debug, GCC 13.3, Ubuntu 24.04:

build runs TSan reports
master 3 1-2 every run
with this change 3 0
#include <atomic>
#include <chrono>
#include <memory>
#include <thread>

#include <fastdds/dds/core/condition/GuardCondition.hpp>
#include <fastdds/dds/core/condition/WaitSet.hpp>

using namespace eprosima::fastdds::dds;
using namespace std::chrono_literals;

int main()
{
    WaitSet wait_set;
    std::atomic<bool> stop{false};

    GuardCondition keepalive;
    wait_set.attach_condition(keepalive);

    std::thread waiter([&]()
            {
                while (!stop.load(std::memory_order_relaxed))
                {
                    ConditionSeq active;
                    wait_set.wait(active, {0, 200000000});
                }
            });

    std::thread churn([&]()
            {
                while (!stop.load(std::memory_order_relaxed))
                {
                    auto gc = std::make_unique<GuardCondition>();
                    wait_set.attach_condition(*gc);
                    gc->set_trigger_value(true);
                    gc.reset();   // destroyed while still attached
                }
            });

    std::this_thread::sleep_for(20s);
    stop.store(true);
    churn.join();
    waiter.join();
    return 0;
}

Note it needs reduced ASLR on 6.x kernels (setarch $(uname -m) -R) or TSan
aborts with "unexpected memory mapping".

Only GuardCondition was exercised by the reproducer. The StatusCondition and
ReadCondition changes are the same pattern applied by inspection: both
destructors were empty, both dereference an impl_ member in
get_trigger_value(), and neither StatusConditionImpl::~StatusConditionImpl()
nor ReadConditionImpl touches the notifier during destruction, so the ordering
of notifier interactions is unchanged.

Contributor Checklist

  • Commit messages follow the project guidelines.

  • The code follows the style guidelines of this project.

  • ❌ Tests that thoroughly check the new feature have been added/Regression tests checking the bug and its fix have been added; the added tests pass locally

    No regression test, because I could not build one that reliably fails on the
    unfixed code inside a sane CI budget, and a test that passes on broken code
    would be worse than none. Measured detection rate with the reproducer above is
    roughly one TSan report per 200k destroy cycles (1-2 per 20s run at ~10k
    cycles/s). Two attempts at a bounded gtest both came up 0/3 against the unfixed
    library: 20000 iterations with one attached condition (~2s/run), and 2000
    iterations with 2000 attached conditions (~25s/run). Widening the pass over
    entries_ makes it worse, because contending on the WaitSet mutex
    anti-correlates the destroying thread with the waiter holding it.

    A deterministic test does not look reachable through the public API either. The
    window is between ~GuardCondition completing and ~Condition starting, and a
    test-local subclass cannot observe it — entering the derived destructor writes
    the vptr before ~GuardCondition runs, so such a test reports a race on fixed
    and unfixed builds alike.

    Happy to add the reproducer to the nightly TSan job instead if you want it
    somewhere, or to take a suggestion on a test shape I have missed.

  • Any new/modified methods have been properly documented using Doxygen.

  • N/A Any new configuration API has an equivalent XML API

  • Changes are backport compatible: they do NOT break ABI nor change library core behavior.

  • Changes are API compatible.

  • N/A New feature has been added to the versions.md file.

  • N/A New feature has been documented/Current behavior is correctly described in the documentation.

  • Applicable backports have been included in the description.

@PavelGuzenfeld
PavelGuzenfeld force-pushed the fix/condition-vptr-race-3883 branch from 142dd66 to 5d6ff3d Compare August 21, 2026 21:14
Condition::~Condition() calls ConditionNotifier::will_be_deleted(), which
runs after the derived destructor has already rewritten the vtable. Until
that call takes the WaitSet mutex the condition is still in entries_, so a
concurrent WaitSetImpl::wait() can call get_trigger_value() on a partially
destroyed object.

Detach in each derived destructor instead, while the object is still fully
constructed. ResourceLimitedVector::remove() returns false for a missing
element, so the later call from Condition::~Condition() is a no-op.

Signed-off-by: Pavel Guzenfeld <me@pavelguzenfeld.com>
@PavelGuzenfeld
PavelGuzenfeld force-pushed the fix/condition-vptr-race-3883 branch from 5d6ff3d to 71ccd94 Compare August 21, 2026 21:27

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

data race on vptr

1 participant